Manipulating Docker in Python (Pull Image, Create/Restart/Delete Container, Port Forwarding)

I wrote this up because I couldn’t find any relatively straight forward answers online that provide an example of consistent usage of Docker in Python.

The code does the following steps:

  1. Checks if the Container Exists:
    • Deletes the Container
    • Restarts the Container
  2. If Container doesn’t exist –
    • Pulls the Image down from Docker Central
    • Creates the Container with specified Port Mappings and Environment Variables
docker_container_name="test"
docker_image_name="tomcat:latest"

docker_internal_port=1234
docker_external_port=1234

deleteDockerContainerEveryTime = True
restartDockerContainerEveryTime = True

def createDockerContainer():
    #create the local instance
    import docker
    client = docker.from_env()

    currentContainers = client.containers.list(all=True)

    containerExists = False

    for container in currentContainers:
        if container.name == docker_container_name:
            if deleteDockerContainerEveryTime == True:
                container.stop()
                container.remove()
                break

            if restartDockerContainerEveryTime == True and deleteDockerContainerEveryTime == False:
                container.restart()
                containerExists = True
                break

            containerExists = True
            break

    if containerExists == False:
        docker_image=docker_image_name

        client.images.pull(docker_image)

        container = client.api.create_container(
                docker_image,
                detach=True, 
                ports=[docker_internal_port, docker_external_port],
                host_config=client.api.create_host_config(port_bindings={docker_internal_port: docker_external_port}, publish_all_ports=True),
                name=docker_container_name,
                environment={
                             'ENV_VARIABLE_1': 'VALUE1',
                             'ENV_VARIABLE_2': 'VALUE2',
                             'ENV_VARIABLE_3': 'VALUE3'
                            })

        currentContainers = client.containers.list(all=True)

        for container in currentContainers:
            if container.name == docker_container_name:
                container.start()
                break

createDockerContainer()

Making API Requests from Shell or Command Line

This article has been copied and reformatted from the below website:

https://hi.service-now.com/kb_view.do?sysparm_article=KB0690780

Description:

  • Scope of this article is to describe how to make a SOAP call to an instance using the CURL command. This will help troubleshooting customer related issues when using the SOAP API.

Step 1:

  • Create the SOAP Envelope that contains the request that has to be sent
  • Note: The soap request parameters depend on the parameters that the WSDL expose.
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:inc="http://www.service-now.com/incident">
   <soapenv:Header/>
   <soapenv:Body>
      <inc:get>
         <sys_id><enter_sys_ID_of_record_to_retrieve</sys_id>
         </inc:get>
   </soapenv:Body>
</soapenv:Envelope>
  • Save it in a file as .xml (example test.xml)

Step 2:

  • Create the CURL command that you will input in the terminal.
    • curl –username:password –header “Content-Type: text/xml;charset=UTF-8” –header “SOAPAction: ACTION_YOU_WANT_TO_CALL” –data @FILE_NAME URL_OF_THE_SOAP_WEB_SERVICE_ENDPOINT
  • Example 1 – If you are consuming the WSDL from an Instance
  • Example 2 – If you are consuming the WSDL from a Node
  • Execute the command in Terminal or command prompt.

Troubleshooting – Tips

  • If ever you see that you are not receiving the expected output, then add the verbose parameter to the curl command and this shall give you more information.
  • Example of the curl command with verbose:

Troubleshooting – Possible Error 1

  • Error – The file containing the request is not readable
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<SOAP-ENV:Fault>
<faultcode>SOAP-ENV:Server</faultcode>
<faultstring>Unable to parse SOAP document</faultstring>
<detail>Error completing SOAP request</detail>
</SOAP-ENV:Fault>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
  • Solution – Check the content and format of the file that contains the SOAP Request

Troubleshooting – Possible Error 2:

  • Error – The credential of the user specified is wrong or user is not authorized to access the WSDL
< HTTP/1.1 401 Unauthorized
< Set-Cookie: JSESSIONID=5C48D05E518DC37DA6502440F2FD8361; Path=/; HttpOnly;Secure
* Authentication problem. Ignoring this.
  • Solution : Check the user credential and if there is any ACL blocking the user from accessing the record

Troubleshooting – Possible Error 3:

  • Error – The WSDL is incorrect/wrong
* Could not resolve host: <URL_OF_THE_SOAP_WEB_SERVICE_ENDPOINT>
* Closing connection 0
curl: (6) Could not resolve host: URL_OF_THE_SOAP_WEB_SERVICE_ENDPOINT
  • Solution: Check if the correct URL is specified in the curl command.

Meeting Reminder Blackout

If you follow this blog you may remember a post a long while back about my “Meeting Reminder Ball” that furiously bounces around the screen.

Well, you may find me crazy but I’ve ironically become numb to it and straight up instinctively close it every time it comes up, effectively ignoring that I may have a meeting because I’m so focused on what I am doing.

BUT NO MORE!

I’ve created a new monster called Meeting Reminder Blackout.

Same process per usual with the Visual Basic Shell call, we launch a C# executable that is pre-written/compiled in Visual Studio and called by Outlook on the Meeting Reminder Event.

Check out the project and installation instructions at my GitHub Repository below:

https://github.com/qwertycody/Meeting_Reminder_Blackout

Tacky Advanced Grepping

From time to time I find myself having to do some quick one off searches to find source code that may have been written already so I don’t have to reinvent the wheel.

In addition, you may remember in a previous blog post or if you have visited my GitHub that I wrote up a Recursive Java Decompiler to allow us to view source code for breakpointing, searching, and much more.

This in hand with the “tacky grepping” has allowed me to find solutions from methods other developers have written and/or usage examples similar to what I’m trying to achieve.

I titled this blog post tacky because I’m essentially piping a pipe of a pipe of a pipe of a pipe of a pipe which is definitely not recommended unless you like turning your computer into a dead potato.

But this is the solution for those of us that either don’t know regular expressions, remember them, or straight up just don’t want to learn them.

#Run in the background
grep -r -i --include \*.java "word1ImLookingFor" | grep -i "word2ImLookingFor" | grep -i -v 'import' > findings.out

#Run in separate Bash Window
#Grep Those Findings For Faster Filtering
grep -i "word3ImLookingFor" findings.out | grep -i -v "word1IdontWant" | grep -i -v "word2IdontWant" | grep -i -v "word3IdontWant" | grep -i "word4ImLookingFor"

Git – Fixing Error – Out of Memory, malloc failed

Saving this for later, but this was my solution for fixing the malloc error. Probably not the best thing in the world that http postBuffer is set to 99999999 but it fixed the issue and I moved on coding.

git config --global pack.threads 1
git config --global core.packedGitLimit 256m
git config --global core.packedGitWindowSize 256m
git config --global pack.deltaCacheSize 256m
git config --global pack.packSizeLimit 256m
git config --global pack.windowMemory 256m
git config --global http.postBuffer 999999999

If Statement in Bash with Multiple Conditions

Took me a little while to get this syntax because bash is super annoying with it’s syntax being different from every other reasonable programming language.

Posting this example so I never have to find it again.

VARIABLE_BOOLEAN="TRUE"

if  [[ -z $VARIABLE_BOOLEAN ]] || [[ ! -z $VARIABLE_BOOLEAN && $VARIABLE_BOOLEAN == 'TRUE' ]]; then
    echo "variable true or null"
fi

WebLogic – Admin Server and Node Manager Utility Script

Wrote this as I wanted any starts of my managed servers under WebLogic to wait until the Admin Server is finished launching.

It also doubles as great syntax bookmark for what core components to start on the WebLogic server and what the filenames are.

function weblogicStart() {
  START_COMMAND="$1"
  LOG_OUTPUT="$2"
	
  "$START_COMMAND" > "$LOG_OUTPUT" 2>&1 &
}

function weblogicStart_ManagedServer() {
  START_COMMAND="$1"
  SERVER_NAME="$2"
  ADMIN_SERVER_URL="$3"
  LOG_OUTPUT="$4"
	
  "$START_COMMAND" "$SERVER_NAME" "$ADMIN_SERVER_URL" > "$LOG_OUTPUT" 2>&1 &
}

function weblogicWait() {
  SERVER_NAME="$1"
  LOG_FILE="$2"

  echo "Waiting until WebLogic $SERVER_NAME is finished starting up before continuing..."
  ( tail -f -n0 "$LOG_FILE" & ) | grep -q "<Server state changed to RUNNING.>"
  echo "WebLogic $SERVER_NAME is Up!"
}

#Log Root for All Logs
LOG_ROOT="/home/oracle"

#Hack to remove any straggling domain backups that prevent changes from persisting on reboot
rm -Rf /home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain/servers/domain_bak/*

#Defining core WebLogic Logs and Commands
WEBLOGIC_DOMAIN_ROOT="/home/oracle/Oracle/Middleware/Oracle_Home/user_projects/domains/base_domain"

WEBLOGIC_ADMINSERVER_LOG="$LOG_ROOT/weblogic.out"
WEBLOGIC_ADMINSERVER_URL="http://localhost:8080"
WEBLOGIC_ADMINSERVER_SERVERNAME="AdminServer"
WEBLOGIC_ADMINSERVER_START="$WEBLOGIC_DOMAIN_ROOT/startWebLogic.sh"

WEBLOGIC_MANAGEDSERVER_START="$WEBLOGIC_DOMAIN_ROOT/bin/startManagedWebLogic.sh"

WEBLOGIC_NODEMANAGER_LOG="$LOG_ROOT/nodemanager.out"
WEBLOGIC_NODEMANAGER_START="$WEBLOGIC_DOMAIN_ROOT/bin/startNodeManager.sh"

#Defining JVM1 WebLogic Server Names and Log Paths
WEBLOGIC_JVM1_LOG="$LOG_ROOT/JVM1.out"
WEBLOGIC_JVM1_SERVERNAME="JVM1"

#Begin Weblogic Commands
weblogicStart "$WEBLOGIC_NODEMANAGER_START" "$WEBLOGIC_NODEMANAGER_LOG"
weblogicStart "$WEBLOGIC_ADMINSERVER_START" "$WEBLOGIC_ADMINSERVER_LOG"

#Wait Until Admin Server is finished booting up
weblogicWait "$WEBLOGIC_ADMINSERVER_SERVERNAME" "$WEBLOGIC_ADMINSERVER_LOG"

#Start Admin Server
weblogicStart_ManagedServer "$WEBLOGIC_MANAGEDSERVER_START" "$WEBLOGIC_JVM1_SERVERNAME" "$WEBLOGIC_ADMINSERVER_URL" "$WEBLOGIC_JVM1_LOG"

SSH Tunnel Script with Commented Usage Example

Useful script for accessing an unexposed port when you only have access to ssh on a server for debug purposes.

########################################
### BEGIN USAGE EXAMPLE FOR COPYING  ###
########################################

##########################################################################
### Usage Example, Copy the below into a separate script and uncomment ###
##########################################################################

#REMOTE_ADDRESS=8.8.8.8
#REMOTE_SSH_PORT=2020
#REMOTE_USERNAME=root
#REMOTE_PASSWORD=password

#declare -A DESTINATION_LIST
#DESTINATION_LIST+=(["5902"]="localhost:5902")

#sh tunnelOpen.sh "$REMOTE_ADDRESS" "$REMOTE_SSH_PORT" "$REMOTE_USERNAME" "$REMOTE_PASSWORD" "$(declare -p DESTINATION_LIST)"

########################################
### END USAGE EXAMPLE FOR COPYING  #####
########################################

REMOTE_ADDRESS=$1; shift
REMOTE_SSH_PORT=$1; shift
REMOTE_USERNAME=$1; shift
REMOTE_PASSWORD=$1; shift
eval "declare -A DESTINATION_LIST="${1#*=}

function establishTunnels()
{
    clear

    for LOCAL_LISTEN_PORT in "${!DESTINATION_LIST[@]}"
    do
        #Pull Values from Array
        DESTINATION_ADDRESS_DESTINATION_PORT=${DESTINATION_LIST[$LOCAL_LISTEN_PORT]}

        echo "Establishing Tunnel to $DESTINATION_ADDRESS_DESTINATION_PORT using Local Port $LOCAL_LISTEN_PORT"

        #ssh-keygen -R [$REMOTE_ADDRESS]:$REMOTE_SSH_PORT
        #ssh -o StrictHostKeyChecking=no -fNL $LOCAL_LISTEN_PORT:$DESTINATION_ADDRESS_DESTINATION_PORT -l root $REMOTE_ADDRESS -p $REMOTE_SSH_PORT >/dev/null 2>&1

        nohup echo y | "/c/Program Files/PuTTY/plink.exe" -N -L $LOCAL_LISTEN_PORT:$DESTINATION_ADDRESS_DESTINATION_PORT -ssh $REMOTE_USERNAME@$REMOTE_ADDRESS -P $REMOTE_SSH_PORT -pw $REMOTE_PASSWORD >/dev/null 2>&1 &

        echo ""
        echo "##################################################################"
        echo "If there are no errors up to this point you can now connect"
        echo "to the endpoint using localhost:$LOCAL_LISTEN_PORT using ssh/rdp"
        echo "##################################################################"
        echo ""
    done
}

establishTunnels

Bluetooth Pairing Script using Expect in Bash

Needed a quick script to do automated pairing in Linux of already known bluetooth devices. I’m using my Switch and L4T Ubuntu to stream using Moonlight and hate having to repair my controllers every time.

Credit to RamonGilabert and the people in the below discussion for their work and iterations. My modification was removing expect of controller (since it wouldn’t work for me), raising the sleep from 5 to 10 on the scanning phase, and adding a connect at the end of the script to force connect the Joycons. Works pretty darn well if I do say so myself.

https://gist.github.com/RamonGilabert/046727b302b4d9fb0055
#!/usr/bin/expect -f

set prompt "#"
set address [lindex $argv 0]

spawn bluetoothctl
expect -re $prompt
send "remove $address\r"
sleep 1
expect -re $prompt
send "scan on\r"
send_user "\nSleeping\r"
sleep 10
send_user "\nDone sleeping\r"
send "scan off\r"
send "trust $address\r"
sleep 2
send "pair $address\r"
sleep 2
send "0000\r"
sleep 3
send_user "\nShould be paired now.\r"
send "connect $address\r"
send "quit\r"
expect eof

Also, as a bonus here is the below Launcher file to be placed on your desktop as “ControllerName.desktop” that does the invocation of the script. Basically just ripped off the Chromium Launcher file.

#!/usr/bin/env xdg-open

[Desktop Entry]
Version=1.0
Name=GreyLeft
Exec=/home/cody/Documents/bluetoothConnect.sh 00:00:00:00:00:00
Terminal=true
X-MultipleArgs=false
Type=Application
Icon=preferences-desktop-remote-desktop
Categories=Network;
MimeType=text/html;text/xml;application/xhtml_xml;x-scheme-handler/http;x-scheme-handler/https;
StartupNotify=true
Name[en_US]=GreyLeft.desktop