Understanding Docker

Docker.md

Understanding Docker

A platform for building, running and shipping applications.

Reasons to use docker

Everyone have stumble upon this case right I works on my machine but not yours? How is that Possible?

Here are the reasons why this happens

1. One or more files missing (Application is not completely deployed).
2. Software version mismatch (If Target machine is using different version of software).
3. Different configuration settings (Like env variables are different accross these machines).

How Docker solves this issue?

Docker will create a package of our application with all of it's dependencies. Like if our App needs Node 14 and Mongo 4. Then it will create a package of Node 14 + Mongo 4 + App and we can ship this package easily anywhere where docker is supported.

Also helps newly hired developer in a company

When a new developer comes in a company. They spend 1-2 days on setting of their machine/env so that project would run smoothly. Now they can use docker and simply run a docker up command and docker will download all the necessary packages, software, dependencies they need to run the project smoothly.

$ docker-compose up

Isolating projects

Docker helps to create different isolated environment for different project. Our old App1 supprots Node 9 and our new App2 supports Node 14. Docker can create separate isolated environment for both of them where one env supports Node 9 and other Node 14. Both these application can run side by side on same machine without messing with each other.

As we work on many projects. Our development machine gets clutter with so many libraries and tools that are used by different application and then after a while we don't know if we can remove one or more of these tools because we are afraid we might mess up with our application.

But with docker we don't have to worry about this. It is easy and safe to remove/delete one container/env with all of its dependencies at once without any trouble because these containers are running in an isolated environment

$ docker-compose down --rmi all

CONTAINER Vs VIRTUAL MACHINE

Container

An isolated environment for running an application.

Virtual Machine

An abstraction of a machine (physical hardware)

Virtual Machine supports isolation and all sorts like what docker does. But Vritual Machine makes use pay more for these features:

1. Each VM needs a full-blown OS.
2. Slow to start (since we are starting entire OS).
3. Resource intensive (takes slice of actual host physical hardware resource).

On the other hand Containers:

1. Are lightweight (they don't need full blown OS).
2. Use OS(Kernel) of the host (all container uses one OS as host these host OS implementation may vary from diff platforms like mac, windows, linux).
3. Start quickly (since host OS is already started launching a container is quick).
4. Need less hardware resources.

Docker commands

1. Command to pull and run ubuntu image from docker hub (ubuntu image will stop immediately)

$ docker run ubuntu

2. To see running docker processes/containers

$ docker ps

3. To see running as well as stopped containers

$ docker ps -a

4. To start a image and interact with it (-it interactive)

$ docker run -it ubuntu

5. Start a container (a2f starting three letters of container id)

$ docker start -i a2f

Creating user in docker image e.g in ububtu

$ docker run ubuntu
$ docker run -it --name test-linux --rm ubuntu

-it interractive, --name custom name for the container, --rm removes container after you are done with it NOTE: It doesnot removes image.

session 1

root@2146e9627282:/# whoami
root@2146e9627282:/# root

root@2146e9627282:/# useradd -m santosh
root@2146e9627282:/# usermod -s /bin/bash santosh
root@2146e9627282:/# cat /etc/passwd
root@2146e9627282:/# cat /etc/shadown

Now while this terminal session is running open another terminal and try to login as new user. Here we are not quitting this session because we want santosh user to persist for this demo. Because if we stop this container then --rm flag wil remove anything related to this container

session 2

$ docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED         STATUS         PORTS     NAMES
a6949b11be48   ubuntu    "bash"    5 seconds ago   Up 4 seconds             test-linux

$ docker exec -it a6949b11be48 santosh bash
sant@a6949b11be48:/$ 

session 1

root@2146e9627282:/# userdel santosh

For deleting user

An interractive way of creating a user (Don't user this command in docker, it is interractive)

$ adduser sant33

Just like for user: useradd, usermod, userdel. We have that for group as well groupadd, groupmod, groupdel.

We can checkout all the groups

$ cat /etc/group

Creating a new group

$ groupadd devs

Adding user to a group

$ useradd -G devs santosh // devs as seconday group of user santosh

or,

$ useradd -g devs santosh // devs as primary group of user santosh

Get a user group name

$ groups $USER

Image

An Image is OS + libs + files + env variables. Let's say it is a class in programming world and container is object.

Container

Isolated process/environment for executing an application which uses it's image file system but (all new containers are isolated by default). Whatever happens in one container doesn't affect other container

var sessionOneLinuxContainer = new UbuntuImage()

var sessionTwoLinuxContainer = new UbuntuImage()

Dockerfile

A Dockerfile contains instructions for building an image.

FROM: Specifying base image. example FROM ubunti:bionic .

WORKDIR: Specifying working dir (after this command next commands will be executed in the dir specified by WORKDIR. example `WORKDIR /home/app/client .

COPY & ADD: Copying files and directories example COPY . /home/app/client .

COPY vs ADD: ADD is COPY on steroids which has additional features like downloading from the internet and untar files.

Best Practice Tip: Use COPY by default. You can use ADD if you know what you are doing.

RUN: For executing os commands. example RUN rm -rf node_modules .

ENV: For setting environment variables .

EXPOSE: Telling docker that our container is staring on a certain port.It does not map ports on the host machine. We need to use -p flag while running a docker image to expose all published ports. example EXPOSE 3000 . EXPOSE is only for documentation.

example:

$ docker run -p -t react-app

USER: Specifying user for running our application by.

CMD & ENTRYPOINT: For specifying commands that should be executed when we start a container .

RUN Vs CMD

RUN: command triggers while we build the docker image CMD: command triggers while we launch/run the created docker image.

Dockerizing a react application

Dockerfile

FROM node:16.13.0-alpine3.14
WORKDIR /home/node/react-app
COPY . .
RUN npm install
CMD ["npm", "run", "dev"]

.dockerignore

node_modules/
README.md

$ docker build -t react-app .
$ docker run --init --rm -p 3000:3000 react-app

Setting environment variables in Dockerfile

ENV API_URL=http://api.app.com/

Dockerfile

FROM node:16.13.0-alpine3.14
WORKDIR /home/node/react-app
COPY . .
RUN npm install
ENV API_URL=http://api.app.com/
CMD ["npm", "run", "dev"]

Check env variables in linux

$ printenv
$ printenv <env-name>
$ echo $<env-name>

Using CMD in Dockerfile

we cannot use multiple CMDs inside a Dockerfile. If we are using multiple CMD, only the last one will take effect.

Also, RUN: Build time instruction CMD: Run time instruction

# Shell form
# docker will run this command in separate shell
# in linux it is /bin/sh
# on windows cmd
CMD npm start # it will create a new process, and it will mess up cleaning process

vs,

# Exec form (Always use this command)
CMD ["npm", "start"] # it will run on same process

CMD vs ENTRYPOINT

CMD ["npm", "start"]
ENTRYPOINT ["npm", "start"]

CMD is pretty flexible we can override it as: docker run react-app echo hello. CMD command will be overriden by echo hello

ENTRYPOINT we cannot easily override command when running a container. It take an effort to override it docker run react-app --entrypoint echo hello.

overriding entrypoint was made this way so that it would be hard for a user to override it because ENTRYPOINT is used whenever we are sure that this is the command that is gonna execute and there is no any exception.

But many prefer CMD because of it's flexibility and easeness.

Optimizing Build time

Docker images are collection of layers. So docker build images by reading instructions from Dockerfile and creates layer after layer. By using this layer docker can determine which instruction are changed and if they are not changed then re-use that layer from cache otherwise re-build it again.

So let's take a scenario heere:

FROM node:16.13.0-alpine3.14
RUN addgroup app && adduser -S -G app app
USER app
WORKDIR /home/node/react-app
COPY . .
RUN npm install
ENV API_URL=http://api.app.com/
EXPOSE 3000
CMD npm run dev

Here FROM node:16.13.0-alpine3.14 will not be changed so let's say - 1 Layer RUN addgroup app && adduser -S -G app app will not be changed so - 1 Layer USER app will not be changed so - 1 Layer WORKDIR /home/node/react-app will not be changed so - 1 Layer

But, COPY . . docker would have to check content of file to verify if it can use cache or rebuild. So if a file is changed then next command would also get rebuild even though we might not require it like for example RUN npm install if we are just changing some logic inside our code we don't need to install packages again. so this will slow down our build.

In order to solve this issue we have to make package installation a separate thing.

Solution

FROM node:16.13.0-alpine3.14
RUN addgroup app && adduser -S -G app app
USER app
WORKDIR /home/node/react-app

COPY package*.json .
RUN npm install

COPY . .
ENV API_URL=http://api.app.com/
EXPOSE 3000
CMD npm run dev

So now unless package*.json file itself haven't been changed we don't have to worry about re-installing things again.

IMP NOTE: TIPS for writing Dockerfile

When writing instructions inside a Dockerfile. Stable instructions should on top and Changing instructions should at lower level.

Clean docker mess

Removing dangling/unconnected images

$ docker image prune

This will not remove anything if those dangling images are used by some containers(stopped or running).If this is the case run below command first then you can remove some dangling images

Removing all stopped containers

$ docker container prune

Removing specific docker image

$ docker images
$ docker image rm <image-name/image-id>

Removing everything

$ docker network prune
$ docker network rm $(docker network ls -q)
$ docker container stop $(docker container list -aq)
$ docker rmi $(docker images -aq) --force
$ docker system prune
$ docker volume rm $(docker volume ls -q)

Tagging images

Whenever we pull or build an images docker tags them as latest. It is bad practice to do this. We have to explicitly tag them in production. latest is fine in case of development but for production or staging you have to follow proper tagging system. Because

1. If something goes wrong without tagging we cannot troubleshoot easily.
2. In future you might have to roll back to previous version in production.

Tagging example

Tagging at build time

We can use code name like buster, or semantice versioning 3.2.1 or build count 16, 17, 18. Build count comes handy incase if you are using CI/CD pipeline.

$ docker build -t react-app:1 .

By doing this we will see that IMAGE ID for every tagged images will be same. But if files are changed then we get different IMAGE ID.

Removing tagged images

$ docker image remove react-app:1

Tagging After build

$ docker image tag react-app:latest react-app:1

Creating "latest version" from last version number

$ docker image tag <latest-version-num-image-id/react-app:2> react-app:latest

Pushing docker images to docker hub

1. Create a repository in docker (e.g. react-app)

2. Now we have uniquely identified image name /react-app e.g. santoshcode/react-app

3. In local tag your latest image to same repo name i.e. santoshcode/react-app

      $ docker image tag <latest-tagged-image-id> santoshcode/react-app   

   This new locally tagged should have or point to same IMAGE ID of latest local version of image.

4. Commands

   $ docker login
   $ docker push santoshcode/react-app:2 # here 2 is what local latest version was so.

5. So let's say you added new feature and changed some file now

      $ docker build -t react-app:3 . # 3 for new version
      $ docker image tag react-app:3 santoshcode/react-app:3
      $ docker push santoshcode/react-app:3

   Next push will be faster if no dependencies are changed.

Sharing and loading images without docker hub.

Saving image in tar file.

$ docker image save -o react-app.tar <image-name-with-tag>

Loading image from tar file.

$ docker image load -i react-app.tar
$ docker images

Starting a container.

See all the running containers.

$ docker ps

Running a container

$ docker run <image-name/repo-name>

Running a container in background/detached mode.

$ docker run -d <image-name>

Giving name to a running container.

$ docker run -d --name tomato-react-app <image-name>

Knowing what is going on inside running docker. i.e LOGS.

Prints out logs from giving container id.

$ docker log <container-id>

To follow up on the logs without typing log command again and again we use -f flag.

$ docker log -f <container-id>

We can use --help flag to see more about this command.

$ docker log --help

Publishing ports

$ docker run -d -p 3000:3000 --name <container-name> <image-name>

Here, in -p flag first 3000 is host machine and second 3000 is container port.

docker exec vs docker run

docker run: We run a new container and run a command.

docker exec: We execute a command and run a container.

Executing command in a running container later on.

Let's see file system of a running container.

$ docker exec <running-container-name> ls 

Let's open running container in interractive mode by creating a shell session.

$ docker exec -it <running-container-name> sh

If you want to open shell as xyz user then

$ docker exec -it -u xyz <running-container-name> sh 

We can exit of this session by using exit command but this will not stop our running container.

docker run vs docker start

docker run: With docker run we start a new container.

docker start: With docker start we start a stopped container.

Stopping and running a container

To stop a container

$ docker stop <container-id/container-name>

To run a container

$ docker start c1

Removing a running container.

Either you stop a container and remove it or you can forcly remove a running container with -f flag.

$ docker rm -f <container-name/id>

Removing all the stopped containers.

$ docker container prune

Volumes

NOTE:: We should never store our data in container's file system because if we remove a container then file system will also get removed as well so for persisting data we should use Volumes in container.

Volumes can be directory in the house or some where in the cloud.

Create volume

We don't have to explicitly create volume before using it. We can do

$ docker run -d -p 4000:3000 -v app-data:/home/santosh/data

This is automatically create app-data volume inside host machine and /data folder inside santosh folder insider container.

$ docker volume create <volume-name>   

example,

$ docker volume create app-data
$ docker volume inspect app-data
[
    {
        "CreatedAt": "2021-11-23T08:34:08+05:45",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/app-data/_data",
        "Name": "app-data",
        "Options": {},
        "Scope": "local"
    }
]
   

Here the Driver is local because we are using host machine to persist our data. If we are going to use colud to persist our data then we need to find suitable driver for it.

Mountpoint is where our data is gonna be. Here in this case it is in our host machine.

For windows mountpoint is gonna be in C drive, For Linux we can use the same mountpoint dir path to see the dir but in case of mac as we know mac uses virtual linux machine to run docker under the hood so when we visit mountpoint path dir in our mac machine then we wouldn't be able to visit it.

Copying between host and containers.

Copying container file to host machine

$ docker cp <container-id>:<container-path-to-file> <host-machine-location>

example,

$ docker cp 3f3b7d93136d:/home/santosh/log.txt .

Copying host file to container

$ docker cp <host-machine-path-to-file> <container-id>:<path-inside-container>

example,

$ docker cp secret.txt 3f3b7d93136d:/home/santosh/data   

Publishing Changes

For production:

Build a new image and tag it properly and push it to production

For Development:

We don't want to build a new image every time we change out code. Also we don't want to manually copy our code to container filesystem.

For Development what we can do is use binding. Binding container to our host source code.

$ docker run -d -p 5001:3000 -v "$(pwd)":<container-source-code-path> react-app

example,

$ docker run -d -p 5001:3000 -v "$(pwd)":/home/santosh/react-app react-app

Here, $(pwd) means hosts source code/project dir and later is where your source code resides inside your container.

Cleaning up

First of all remove all the containers

$ docker container rm -f $(docker container list -aq)   

Here, -a flag refers to stopped containers and -q flag provides us with all the container ids.

Removing all the images

$ docker rmi -f $(docker images -aq)   

Removing volumes

$ docker volume rm $(docker volume ls -q)