Docker notes

Docker_Notes.txt

Notes :
https://www.udemy.com/docker-mastery/
https://www.youtube.com/watch?v=EnJ7qX9fkcU
https://github.com/matt9ucci/DockerCompletion

why docker
speed
reduce the complexity
Is docker just container run-time ?

Docker Editions
docker is no longer just a " container runtime"
CE vs EE
three major types of installs\
direct -> it means it runs on that supported OS kernel
Mac/Win -> suit of tools including GUI/settings. Basically what happens in the background is a small virtual machine has to be started by docker to run the containers in. Docker isn't natevly supported in Mac/Win
cloud -> AWS/Azure/Google.. comes with different options

# Windows docker options
    - two types, linux container and Windows containers
    - the difference is the binaries are running
    - in this course, Linux containers
    - on Win 10, uses Hyper-V with tiny linux VM to run Linux containers
    - powerShell native
    - for docker cli complion on powerShell, please check https://github.com/matt9ucci/DockerCompletion

# Check our Docker install and Config
    - containers are the fundamental building block of the docker toolkit
    - cm : docker version
    - cm : docker info
    - mangement commands vs commands (both work)
    - old commands vs new commands (docker run vs docker container run)
    - old command -> docker <command> (option)
    - new management command -> docker <command><sub-command> (option)
  
# Image vs Container
    - image -> is the binaries, libs and source coud that make your application that you want to run
    - container -> is an instance of the image running as process
    - you can have many containers running off the same image
    - image registry

# run Nginx container
    - cm : docker container run --publish 80:80 nginx
        1. download image 'nginx' from docker hub
        2. start a new container from that image
        3. open port 80 on the host IP (left port) ** you can use any port **
        4. routes that traffic to the container IP, port 80
        
    - cm : docker container run --publish 80:80 --detach nginx ( <detach> tells docker to run in the background)
    - cm : docker container ls (only shows running containers)
    - cm : docker container ls (only shows all containers)
    - cm : docker container stop <container-id>
    - cm : docker container run --publish 80:80 --detach --name webhost nginx
    - cm : docker container logs webhost
    - cm : docker container top webhost (top processes inside the container)
    - cm : docker container rm -f 3ede76cf686f (-f to force container to stop)
    - cm : docker rm $(docker ps -aq)

# What happens when we run a container
    - look for the image locally in image cache, doesn't find anything
    - then looks in remote image repo
    - dowlanod the latest version if we do not specify a version
    - creates a new container based on that image
    - gives it a virtual IP on a private network inside docker engine
    - opnes up port on host and fowwards to port in container if we specify those ports
    - starts container by using the commands in the image Dockerfile


# Containers vs Virtual machines
    - containers are just processes inside the host system
    - Limited to what resources they can access
    - Exit when process stops

# Linux containers vs Windows containers
    - https://www.youtube.com/watch?v=066-9yw8-7c
    - https://www.youtube.com/watch?v=4ZY_4OeyJsw
    - https://www.youtube.com/watch?v=QASAqcuuzgI


# Assignment : Manage Multiple Containers
    - cm : docker container run -d -p 3306:3600 --name mysql -e MYSQL_RANDOM_ROOT_PASSWORD=true mysql
            what --> create container "db" from lastes image of "mysql", and publish it on port 3306, and
                        get env vatiable MYSQL_RANDOM_PASSWORD

    - cm : docker container logs db
            what --> capture -e value

    - cm : docker container run -d --name webserver -p 8080:80 httpd
            what -> create container "webserver" from lastes image of "httpd", and publish it on port 8080:80

    - cm : docker container ps (old command)

    - cm : docker container run -d --name proxy -p 80:80 nginx

    - cm : docker container rm -f a4e71859540e 88bff017b331  8ef5227cb704
         what -> delete all containers and forec them to stop before deleting

# What's going on in containers
    - cm : docker container top (process list in one container)
    - cm : docker container inspect (details of one container config)
    - cm : docker container stats (perofrmance stats for all containers)

# Getting a shell inside containers
    - cm : docker container run -it (start new container interactively)
    - cm : docker container exec -it (run additional command in exisiting container)

    - cm : docker container run -it --name proxy nginx bash
            what -> start container in interactive mode and start  bash on startup

    - cm : docker container run -it --name ubuntu ubuntu
    - cm : docker container start -ai ubuntu (start an existing container in interactive mode)  
    - cm : docker container exec -it mysql bash (start interactive insidign running container)

# Docker networks : Concepts
    - cm : docker container run -p (expose ports)
    - docker has the comncept of batteries included but removable which basically means the defualts
        are pretty easy and common to use, but you can change those defualts.
    - cm : docker container port <container> (list open ports on your container)

# Docker networks defaults :
    - each container conncted to a private virtual network "bridge"
    - each virtual netwrok routes through NAT firewall on host IP .
        - basically Docker daemon configuring the host IP adresss on its default interface so that the containers get out
          to the internet or to the rest of the network.
    
    - All containers on a virtual network can talk to each other without -P
    - best practice to create virtul network for each app. for example :-
        - network "my_web_app" for mysql and php/apache containers
        - network "my_api" for mongodb and node.js containers
    - cm : docker container port webhost
    - cm : docker container inspect --format '{{ .NetworkSettings.IPAddress }}' webhost (to get container IP)
           - Notice how container IP is different from host IP

# How Docker networks move packages  in and out :
    - my host is connected to my network
    - assuming it's a physical network
    - we have Ethernet interface on my host machine
    - there's a firewall on that interface that does a lot of stuff, for example, it bloacks all incoming traffic from network
      so by defualt everthing is bloacked.
    - Any traffic coming from my container would be NAT-ed by default.
    - there's a concept called virtual network and by default you'll see network called "bridge" or "Docker0"
    - when you start new container "container_1", the container is attached to the "bridge" network. and that virtual network
      is automatically attahced to our Ethernet interfaceon our host.
    - on any interface on your host, you can't listen on more than one port for Multiple containers.

# Docker networks: CLI management
    - cm : docker network ls (show networks)
    - cm : docker network inspect (inspect a network)
    - cm : docker network create --network (create a network)
    - cm : docker network connect (attache a network to a container)
    - cm : docker network disconnect (deattahce a network from a container)

    - there's a spcial network called "host" when you do (docker network ls) it skips the virtual network of Docker
      and attahce the container directly to the host interface. therte are pros/cons

    - cm : docker network create my_app_nt
    - cm : docker container run -d --name my_nginx --network my_app_nt nginx (create a new container on a speific network)
    - cm : docker network connect 7bedb818377c 7270f26fd7f2 ( attahced a newwork to a container, now our container is attached to 2 networks)
    - cm : docker network connect <network> <container> ( attahced a newwork to a container )
    - cm : docker network disconnect 7bedb818377c 7270f26fd7f2 ( deattach a newwork from a container, now our container is attached to 1 networks only)


# Docker networks: DNS
    - understand how DNS is the key to easy inter-container comms. we can't rely on IP inside
      containers since things are dynamic.
    - difference between default and customer networks in regards to hwo they deal with DNS
    - --link to enable DNS on default bridge network
    - Static IP's and using IP's for talking to containers is an anti-patern.
    - there's built-in soltuion for DNS naming, and docker uses the container names
      as the equivalent of a host name for containers talking to each other.
    - bridge network does not hae DNS server built-in into it by default so you need to use --link
    - by default, when you create new container it will be attahced to a virtual newwork called "bridge" unless you specify different network

# what's a container images and what's not
    - app binaries and dependencies
    - metadata about the image data and how to run the image
    - Not a complete OS. No kernel, no kernel modules (e.g. drivers)
    - it's just the binaries that your app needs because the host provides the kernel. this one the distinct charas around
      containers that makes it different from virtual machines, becuase container just runs the app doesn't
      boot a full OS like VM

# Dokcer hubs, images :
    - cm : docker pull nginx

# Image lyers :
    - image lyers
    - union file system
    - history and insoec t commands
    - copy on write ***
    - images are designed using the union file system concept of making layers about the changes.
    - cm : docker image ls
    - Cache concept in images. we don't need to downlaod layers we already have.
    - Each layer has it's identiy as SHA (uniquely identified ) and only stored once on a host ( on docker deamon )
    - a container is just a single read/write layer on top of image !!!
    - cm : docker image history nginx ( to see changes on the image over time)
    - cm : docker image inspect nginx ( show metadata )

# Image tagging, pushing to docker hub
    - cm : docker image tag nginx matar86/nginx ( tag an existing image )
    - cm : docker login
    - cm : docker image matar/nginx push

# the Dockerfile basics


# Buidling images : running docker builds
    - cm :  docker image build -t customnginx . ( build docker image "customenginx" from Dockerfile in the current dir)
    - docker gives each line in Dockerfile uniqe hash whichs very useful when we build same image again. Basiclly any unchange line
      will be skipped in next build which will save a lot of time and space.
    
    - let expose one more port in our Dockerfile then build new images.
    - cm :  docker image build -t customnginx . ( now, it will use cache for each unchanged line)

# Buidling images : extending offical images :
    - cm : docker image build -t nginx-with-html .

# build your docker image  :
    - cm : docker image build -t testnode .
    - cm : docker tag testnode matar86/testing-node
    - cm : docker push matar86/testing-node
    - cm : docker image rm matar86/testing-node (to remove it from local repo)
    - cm : docker container run --rm -p 80:3000 matar86/testing-node ( pull it from docker hub)



# Container lifetime and persistent data :
    - defining the problem of persistent data
    - key concepts with containers -> immutable, ephemeral
    - Data volumes
    - Bind mounts

    - Containers are usually immutable and ephemeral -->
        it means they are unchanging and temporary o disposal.
        the idea here is that we can throw away a container and create a new one from an images.
        so it's not an actuial Limition of containers, but more of a desgin goal or best practice.
        this is the idea of immutable infrasturcte where we don't change this once they're running. if config
        chnage needed or maybe upgrade the container then we redeploy the whole containers. but there's a trade off
        what about the data app produces or databaes ?
        Ideally, the containers should not contain your unique data mixed in with the app binariies. This is known as
        " separation of concerns "

        -- unique data problem which is know as " presistant data ", when we remove container data isn't persistance and will be lost.
            Docker has 2 uniqe solutions for this problem ->
            1. volumes -> docker volumes are a config options for a contanier that creates specail locatopn outside
                        the container's union file system to store unique data. it will just look like a local file path or dir path to the container
            2. Bind mounts -> are simply us sharing or mounting a host directory or file into a containers.
                              and it will just look like a local file path or dir path to the container.


# Persistent data : Data volumes
    - first way to tell contaier about "vloumes" is via VOLUME cmd in Dockerfile.
    - cm : docker volume ls
    - when we remove containers, vlumes don't get removed. We need to remove them manually.
    - nameed valumes --> frienly way to assign volumes to containers.
    - cm : docker container run -d --name mysql -e MYSQL_ROOT_PASSWORD=True -v mysql-db:/var/lib/mysql mysql
            (start the container with named volume)
    - cm : docker volume ls
    - cm : docker volume create ... (we use it to create docker vlome ahead of time, it's not common )
    
# Persistent data : Bind Mounting
    - maps a host file or dir to a continaer file or dir
    - basically 2 locations poiting to the same file(s)
    - skpis UFS and host files overwrite any in container.
    - can't use in Dockerfile, must be at container run
    - cm : run -v //c/users/bret/stuffL/path/container (Windows)
    - cm : docker contaier run -d --name nginx -p 80:80 -v $(pwd):/user/share/nginx/html nginx
    

# docker Compose :
    - combination of command line tools and config file
    - To config relationships between containers
    - save our dcoker container run settings in easy-to-read file
    - create one-liner dev env startup
    - Docker Compose comprised of 2 separate but related things :->
        - YAML-formatted file that describe solution options for
            - containers
            - networks
            - volumes
            - images, env variables
            - etc
        - A CLI tool docker-compose used for local dev/test automation with those YAML files.

# docker-compose.yml :
    - compose YAML format has it's own versions 1,2, 1.2, and so on
    - YAML file can be used with docker-compose commmand for locakl automation
    - with Docker directly in prod with Swarm (as of V1.13, 2017)
    - cm : docker-compose
    - docker-compose.yml is defualt filename, but any can be used with docker-compose -f

    - example 1
        # same as
        # docker run -p 80:4000 -v $(pwd):/site bretfisher/jekyll-serve
        version: '2'
        services:
        jekyll:
            image: bretfisher/jekyll-serve
            volumes:
            - .:/site
            ports:
            - '80:4000'
    
    - example 2 :-> Multiple comtainers(services)
        version: '2'

        services:

        wordpress:
            image: wordpress
            ports:
            - 8080:80
            environment:
            WORDPRESS_DB_HOST: mysql
            WORDPRESS_DB_NAME: wordpress
            WORDPRESS_DB_USER: example
            WORDPRESS_DB_PASSWORD: examplePW
            volumes:
            - ./wordpress-data:/var/www/html

        mysql:
            image: mariadb
            environment:
            MYSQL_ROOT_PASSWORD: examplerootPW
            MYSQL_DATABASE: wordpress
            MYSQL_USER: example
            MYSQL_PASSWORD: examplePW
            volumes:
            - mysql-data:/var/lib/mysql

        volumes:
        mysql-data:
            

# docker-compose CLI
    - cm : docker-compose up
    - cm : docker-compose down


# Adding image build to compose files
    - docker-compose can build your images at runtime
    - will build them with docker-compose up if not found in cache
    - rebuild with docker-compose build
        
    - example (https://github.com/BretFisher/udemy-docker-mastery/tree/master/compose-sample-3)
        version: '2'
        services:
        proxy:
            build:
                context: .
                dockerfile: nginx.Dockerfile
            ports:
            - '80:80'
        web:
            image: httpd
            volumes:
            - ./html:/usr/local/apache2/htdocs/

# docker-compose assignment 1/2 (https://github.com/BretFisher/udemy-docker-mastery/tree/master/compose-assignment-2):
    - cm : docker-compose up
    - cm : docker-compose down -v
    - cm : docker-compose down --rmi all

    - solution :-> check the (docker-compose.yml and Dockerfile) in the repo



# Swarm Mode : Built-iin Orchestration
### I will skip Swarm part becuase I want to lear Kubernetes instead
    - how can we easily deploy and maintain our continers across tons of servers and instances ?
    - how do we automate containers lifecycle ? (deployment, startin, restarting, upgrading, etc)
    - how we scale out/in/up/down ?
    - how we handle blue/green deploymwnt ?
    - cross-nde network ?

    - Swarm mode is clustering solution built inside docker
    - not related to Swarm "classic"
    - Swarm mode is not enable by defualt