Docker cheat sheet

Docker cheat sheet

#getting an specific image
docker pull ubuntu

#A container is a running image that you start with the docker run command, like this:
$ docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

#We then run a container from that image:
docker run -it ubuntu /bin/bash #And that’s it, root@4ff0be4995f0:/# means that we are root inside an Ubuntu container 
/
#Now, if we want to assign a name to the container created from the ubuntu image
docker run -it --name="simple_flask" ubuntu:latest /bin/bash
//
#Automatically remove the container when it exits
docker run -rm
//
# stop and remove all containers
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
//
#listing all images
docker images
//
#if we want to give a name (simple_flask in this case) to a certain image:
docker tag 63b833218f99 simple_flask
#Where 63b833218f99 is the image id
//
#listing all images, including the inactive ones
docker images -a
//
#removing a certain image
docker rmi imagename
//
#removing all images
docker rmi $(docker images -a -q )
//
#if when we do:
docker images
#we get:
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
ubuntu                latest              ccc7a11d65b1        18 hours ago        120MB
#We can delete ubuntu by doing:
docker rmi ubuntu:latest
//
Create a container with the name test-mysql. If you don’t specify this, Docker will generate a random name.
docker run --name=test-mysql mysql
//
#create a container with name my-redis and -it allows to interact with it from the command line. 
docker run --name my-redis -it ubuntu:latest bash
/
#Run a binary inside a container and mounting a file used for the binary
docker run --volume /Users/ernesto/DOCKERFOLDER/input10.reheaded.vcf.gz:/input10.reheaded.vcf.gz image_name bcftools view -h /input10.reheaded.vcf.gz
#Where '/Users/ernesto/DOCKERFOLDER/input10.reheaded.vcf.gz' is the location in the local system and '/input10.reheaded.vcf.gz' is the file mounted in the container
/
# Another way of mounting the file in order to use within the container the same path than in the local filesystem is:
 docker run --volume $HOME:$HOME --workdir $PWD variant_filtering  bcftools view -h /Users/ernesto/DOCKERFOLDER/input10.reheaded.vcf.gz
 # Where '/Users/ernesto/DOCKERFOLDER/input10.reheaded.vcf.gz' is the path in the local filesystem
/
#once you are inside a container, you can exit by typing:
exit
//
*set an environment variable from the command line
docker run --env="MYSQL_ROOT_PASSWORD=mypassword"
//
*running a container in the background, in this case a mysql container
 docker run --detach --name=test-mysql --env="MYSQL_ROOT_PASSWORD=mypassword" mysql
//
#executing a command in a certain image (in this case, echo is executed within the ubuntu image)
docker run ubuntu echo "hello world"
//
#see the contents of an image
docker run imagename ls
//
#tutorial on how to create an image with a container that executes a python script:
https://www.civisanalytics.com/blog/using-docker-to-run-python/
//
*Using anaconda with docker:
docker pull continuumio/anaconda3

and now we can start an interactive shell:
docker run -i -t continuumio/anaconda3 /bin/bash
/
#in general, when we want to run an interactive shell, then we do:
docker run -i -t test_bwa /bin/bash
#with -t imagename we run an especific imagename
//
#build an image and specify the docker file
docker build - < DockerFilename

or

docker build -t jenkins-docker-image -f JenkinsDockerfile .
/
#storing output from build to file
docker build - < DockerFilename > build.log
/
//
#replacement of $PATH env variable in a dockerfile
ENV PATH="/opt/gtk/bin:${PATH}"
//
#Dockerfile. Add a file from host to container
#where /home/variants.chr20.vcf.gz is the location of the file in the container
ADD variants.chr20.vcf.gz /home/variants.chr20.vcf.gz
//
#In a dockerfile, we can change to a specific folder by doing:
WORKDIR foldername
//
#In a dockerfile, we can specify the latest ubuntu image by doing:
FROM ubuntu:latest
//
#In a dockerfile, run a certain command:
RUN apt-get update
#where apt-get update is the command we want to run
//
#to list all running containers
docker ps

This command will output something like:
CONTAINER ID        IMAGE                COMMAND                  CREATED             STATUS              PORTS                    NAMES
acbc50d45d40        ubuntu:latest        "/bin/bash"              50 seconds ago      Up About a minute                            zen_nobel

And if we enter:
docker diff zen_nobel

We get all the changes that we have made to the container

And now we can commit the changes to the image:
docker commit -m "Adding hello world" zen\_nobel hello\_docker

Now, if we enter 'docker images'. We will see:
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
hello_docker             latest              65adb3dc10b2        38 seconds ago      84.1MB

And if we want to see the history of this image we can do:
docker history hello_docker
//
#show all containers (including the non-running ones):
docker ps -a
//
#now, if we want to stop a container we do:
docker stop acontainer_id
#where container_id is obtained after running 'docker -ps -a'
//
# if we want to remove all stopped containers:
docker container prune
//
#if we want to delete the container:
docker rm acontainer_id
//
# if we want to remove all container ids that are inactive. We can do:
docker rm `docker ps --all -q`
//
*Check how to run a mysql docker image and read:
https://severalnines.com/blog/mysql-docker-containers-understanding-basics
/
*Run it on the background
docker run --detach --name=test-mysql --env="MYSQL_ROOT_PASSWORD=mypassword" mysql
/
*Check the container logs:
docker logs test-mysql
/
*In order to connect to the mysql-server container we have to look for the IPAddress that will
be used to connect with the mysqlclient
docker inspect test-mysql |grep IPAddress
/
Also, we can check the mysql root password:
docker inspect test-mysql |grep password
//
#docker inspect can also be used to inspect a certain image:
docker inspect dd75ba3c81a5 
#Where dd75ba3c81a5 is the image id
//
* To push an image to docker hub:
* First you need to login to dockerhub
> docker login
* Tag your local image:
docker tag variant_filtering elowy01/variant_filtering:latest
* Then, you can push the image to the repository, the image should be tagged in order to match the docker username and the 
* repository name:
docker push elowy01/variant_filtering:latest
//
#In order to pull an image from docker hub:
docker pull elowy01/variant_filtering:latest
//
#knowing how much memory docker is using
 docker system df
//
#cleaning (prune) unused stuff
docker system prune
//
#How can I inspect the file system of a failed `docker build`?
Read post at Stackoverflow:
https://stackoverflow.com/questions/26220957/how-can-i-inspect-the-file-system-of-a-failed-docker-build/35387446#35387446
//
# Accessing MySQL using a web interface through docker (Read post at https://medium.com/coderscorner/connecting-to-mysql-through-docker-997aa2c090cc)
//
# Installing Jupyter notebook

1) Run 
docker pull jupyter/datascience-notebook:latest

2) Run the container
docker run -p 8888:8888 --name jupyter jupyter/datascience-notebook:latest

And write down the url and token printed in the screen

Other option:
docker run -d -p 8888:8888 --name jupyter \
--env JUPYTER_TOKEN=admin \
--volume "/Users/ernesto/notebooks/":/home/jovyan/work jupyter/datascience-notebook:latest

There are several things going on here:
-We are setting the token to what we want, in this way we can run it in detach mode.
-Also, we are mapping our local host volume to a folder in the container, in this way
we can copy a notebook to /Users/ernesto/notebooks/ and after refreshing the web we will
see the jupyter nb.
//

The web will be accessible using:
http://localhost:8888/?token=jupyter_notebook_token
/
# Now, imagine that you try to import a dependency (i.e. tensorflow) in your
# running container. It will complain. For solving this you need to install
# the dependency in the running container by first getting the container id:

> docker ps  
And you will get:
CONTAINER ID        IMAGE                                 COMMAND                  CREATED             STATUS              PORTS                    NAMES
a9585028d118        jupyter/datascience-notebook:latest   "tini -g -- start-no…"   7 days ago          Up 7 days           0.0.0.0:8888->8888/tcp   jupyter
Then, you install tensorflow by using pip in the running container by doing:
docker exec a9585028d118 pip install tensorflow

#The problem of this approach is that this dependency will be lost when you stop
the running container and you will need to install it again the next time you
run the container

//
# How to list the child images:
for i in $(docker images -q)
do
    docker history $i | grep -q f50f9524513f && echo $i
done | sort -u
#where f50f9524513f is the parent image
//
##############
# Differences btw CMD and ENTRYPOINT (extracted from https://blog.codeship.com/understanding-dockers-cmd-and-entrypoint-instructions/)
First, let's show the usage of CMD. We create the following image that uses the 'ab' command:

FROM ubuntu:latest

RUN apt-get update && \
    apt-get install -y apache2-utils && \
    rm -rf /var/lib/apt/lists/*

CMD ab

# if we build it:
docker build - -t ab <test_df.txt
# and we run it with:
docker run ab
# we get the help of ab because we have not args:
ab: wrong number of arguments
Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
    -n requests     Number of requests to perform
    -c concurrency  Number of multiple requests to make at a time
    -t timelimit    Seconds to max. to spend on benchmarking
                    This implies -n 50000
    -s timeout      Seconds to max. wait for each response
                    Default is 30 seconds
    -b windowsize   Size of TCP send/receive buffer, in bytes
    -B address      Address to bind to when making outgoing connections
    -p postfile     File containing data to POST. Remember also to set -T
    -u putfile      File containing data to PUT. Remember also to set -T
    -T content-type Content-type header to use for POST/PUT data, eg.
                    'application/x-www-form-urlencoded'
                    Default is 'text/plain'
    -v verbosity    How much troubleshooting info to print
    -w              Print out results in HTML tables
    -i              Use HEAD instead of GET
    -x attributes   String to insert as table attributes
    -y attributes   String to insert as tr attributes
    -z attributes   String to insert as td or th attributes
    -C attribute    Add cookie, eg. 'Apache=1234'. (repeatable)
    -H attribute    Add Arbitrary header line, eg. 'Accept-Encoding: gzip'
                    Inserted after all normal header lines. (repeatable)
    -A attribute    Add Basic WWW Authentication, the attributes
                    are a colon separated username and password.
    -P attribute    Add Basic Proxy Authentication, the attributes
                    are a colon separated username and password.
    -X proxy:port   Proxyserver and port number to use
    -V              Print version number and exit
    -k              Use HTTP KeepAlive feature
    -d              Do not show percentiles served table.
    -S              Do not show confidence estimators and warnings.
    -q              Do not show progress when doing more than 150 requests
    -l              Accept variable document length (use this for dynamic pages)
    -g filename     Output collected data to gnuplot format file.
    -e filename     Output CSV file with percentages served
    -r              Don't exit on socket receive errors.
    -m method       Method name
    -h              Display usage information (this message)
    -I              Disable TLS Server Name Indication (SNI) extension
    -Z ciphersuite  Specify SSL/TLS cipher suite (See openssl ciphers)
    -f protocol     Specify SSL/TLS protocol
                    (SSL2, TLS1, TLS1.1, TLS1.2 or ALL)
# and if we try to use it with an argument we need to use a solution that is not very elegant:
docker run ab ab http://bencane.com/

# The solution for this is to use ENTRYPOINT:
# We rewrite the image to:
FROM ubuntu:latest

RUN apt-get update && \
    apt-get install -y apache2-utils && \
    rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["ab"]

# and if we build it again:
docker build - -t ab <test_df.txt

# we can run it by doing:
docker run ab http://bencane.com/
################
//
# If we create a virtual package in a DF named .build-deps, this 
# virtual package will not be global and can contain the dependencies
# we want and we can delete them when we longer no need them so
# we do not increase the size of the image

RUN apk add --no-cache --virtual .build-deps \
gcc \
freetype-dev \
musl-dev

RUN pip install --no-cache-dir <packages_that_require_gcc...> \

RUN apk del .build-deps
//
# Run several containerized programs in a pipe
docker run --volume $HOME:$HOME --workdir $PWD elowy01/igsr_analysis /bin/bash
-c "bcftools view data/test.vcf.gz.filtered.vcf.gz | bcftools view -H"

# copying a file from container to local machine
docker cp igsr_analysis_container:/lib/igsr_analysis/tests/reports/result.xml .

# start a stopped container
docker start  `docker ps -q -l`
docker attach `docker ps -q -l`