Python machine learning libraries in docker

ml-Python-libraries.md

Machine learning libraries of Python in Docker

The docker tar.gz'd is 2.4 GB in size, download from here

background and installed libraries in the docker

• to set up the new ubuntu docker image, it is 124MB in size,

  sudo docker search ubuntu
  sudo docker pull ubuntu
  sudo docker run -dit ubuntu bash
  sudo docker exec -it 039fc067339c bash
  apt-get update
  apt install wget
  wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
  apt install bzip2
  bash Miniconda3-latest-Linux-x86_64.sh
  which python
  python --version
  

• installing libraries needed for ML

conda install -c conda-forge pandas
conda install -c conda-forge dlib
apt-get update
apt-get install -y build-essential
apt install cmake
pip install face_recognition
pip install imutils
conda install -c conda-forge keras
pip install opencv-contrib-python
pip install opencv-python
conda install -c conda-forge scikit-image
conda install -c conda-forge scikit-learn 
pip install tesseract
pip install urllib3
pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.8.0-cp36-cp36m-linux_x86_64.whl
for import cv2 error
apt update && apt install -y libsm6 libxext6
apt-get install -y libxrender-dev

• testing the libs, keras

from keras.models import Sequential
from keras.layers import Dense, Activation

which ends up in error saying no tensorflow which was solved by installing tensorflow as per here

• testing the tensoflow

import tensoflow

no error

• testing the opencv

import cv2

ends up in error as per this and this, solved it, there is good nte on keras, cv2 installation here

• commit the container

sudo docker commit -a "nishadh" -m "ML tools-Keras,tf,cv2" 039fc067339c ubuntu-ml-py36/ubuntu-ml-py36:version1

• there was an error in matplotlib and added the jupyter, geopandas, rasterio and moviepy

conda install -c conda-forge geopandas 
conda install -c conda-forge jupyter
dpkg --add-architecture i386 
apt-get update
apt-get install libsm6 libxrender1 libfontconfig1
conda install -c conda-forge basemap
conda install -c conda-forge python-fmask
conda install -c conda-forge rasterio
conda install -c conda-forge moviepy
#ffmpeg is further download upon improting the moviepy first time
#utility funcitons for ml based [on](https://arxiv.org/abs/1712.00321)
conda install mlxtend
conda install -c conda-forge holopy

• commit again the container

sudo docker commit -a "nishadh" -m "ML tools-v2" 039fc067339c ubuntu-ml-py36/ubuntu-ml-py36:version2

• added the zip for nomral unzip and holopy

apt-get install zip
conda install -c conda-forge holopy 

• commit again the container

sudo docker commit -a "nishadh" -m "ML tools-v3" 039fc067339c ubuntu-ml-py36/ubuntu-ml-py36:version3

• saving the docker

docker save e94f7de5b7d9 'ubuntu-ml-py36/ubuntu-ml-py36:version3' > ubuntu-ml-py36_v3.tar
tar -cvzf ubuntu-ml-py36_v3.tar.gz ubuntu-ml-py36_v3.tar

• this is the tar.gz available in above link

to run the Jupyter server from ml docker in Ubunut

• to install docker in ubuntu

  sudo apt-get update
  sudo apt-get install apt-transport-https ca-certificates curl software-properties-common
  curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
  sudo apt-get update
  sudo apt-get install docker-ce
  

• to load the docker image, to start, unzip the image ubuntu-ml-py36_v3.tar.gz into ubuntu-ml-py36_v3.tar, then do

  docker load -i ubuntu-ml-py36_v3.tar
  

• then get to know the image name, image_id by docker ps and then execute

  docker run -dit -p 8888:8888 ubuntu-ml-py36/ubuntu-ml-py36:version3 bash
  docker exec -it $get_id$ bash
  jupyter notebook --ip 0.0.0.0 --no-browser --allow-root
  

• then know the ip address of running docker image from the host and use the url with the ip address to get into the jupyter notebook from the docker image

to run the Jupyter server from ml docker in Windows

• setup the docker toolbox in Windows 7/8/10, docker app of windows was not tested in following method
• Have local copy of Docker toolbox from here for windows 64 bit, for 32 bit Windows, follow this link
• Have local copy of boot2docker.iso from here
• Save the iso file in C:\Users\User.docker\machine\cache\boot2docker.iso
• Open the program Docker quick start, It will make a seprate virtual machine to run the docker conatiner
• Under the Docker quick start program, after the virtual machine run, check docker is working by '''docker ps'''
• Now, import the image ubuntu-ml-py36_v3.tar

  docker load -i ubuntu-ml-py36_v3.tar
  

• Then run the imported container image by

  docker run --name ubuntu-ml --rm -it --user root -d -p 8080:8080 ubuntu-ml-py36/ubuntu-ml-py36:version3
  

• again check the command docker ps, it would show the container is running by having a container ID
• To get into the running docker

  docker exec -it container_ID bash
  jupyter notebook --ip=0.0.0.0 --port=8080 --allow-root
  

• typical url of Jupyter notebook is http://0.0.0.0:8080/?token=431388d3beed502223f3ae00ddb807cdd373f1aa97e6c68f