Installing Deep Learning tools on a cloud instance

aws_deeplearning_ami.md

Launch an EC2 Instance

0. Check Spot Instance Pricing here
1. Open EC2 console
2. Choose AMI :
   • Deep Learning AMI (Ubuntu) Version 8.0 - ami-dff741a0
     • contains TensorFlow and other famous DL libraries
     • make sure you select the "Ubuntu" version and not "Amazon Linux" version
3. Instance Type : Select an instanct type from the p2/p3 family
   • For inference purposes - p2.xlarge (1 x Nvidia Tesla K80)
   • For training purposes - p3.xlarge (1 x Nvidia Tesla V100)
4. Configure Instance :
   • Check Request Spot instances
5. Add Storage :
   • For inference purposes - Select 150GB (Magnetic) -- keeps cost low
   • For training purposes - Select 500GB+ (SSD) -- keeps read times low
   • You may check / uncheck - "Delete on Termination"
6. Add Tags :
   • Always add tags as its helps while using the Bill Explorer
   • Simple option is {'project' : <project-name>}
7. Configure Security Group :
   • Open ports for
     • 22 : SSH
     • 5000 : Flask Server
     • 6060 : Tensorboard
     • 8888 : Jupyter Notebook
8. Review :
   • Do take 2-3 mins to go through this.
9. Key-Pair :
   • Either create a new .pem file or use an existing .pem file

Login to an EC2 Instance

1. Go to your EC2 Dashboard
2. Copy the IPV4 Public IP
3. Open a Command Prompt

• ssh -i .pem ubuntu@

Post-Login

1. Setup Tensorflow
   • for TensorFlow(+Keras2) with Python3 (CUDA 9/MKL) ____________ source activate tensorflow_p36
     • First activation might takes some time (3+ mins)
     • Make a note of the library versions (for future references) in a .txt file
   • Once installation is done, the command line shall be prepended by the environment name (tensorflow_p36)
2. Some Basic Checks
   • Check Nvidia Driver Version - nvidia-smi
   • Check GPU - CUDA :
     • cd /usr/local/cuda-9.0/samples/1_Utilities/deviceQuery
     • sudo make -j4 deviceQuery
     • pyth --> Detected 1 CUDA Capable device(s)
   • Check default Python version
     • Type python --> Check top lines of the interpreter tht opens up
   • Check pip version
     • pip --version
3. Install all the requisite libraries for your application
   • Imaging Libs
     • pip install opencv-python Pillow scikit-image
     • pip install shapely descartes matplotlib
   • Python - Web Server Libs
     • pip install flask
   • Storage Libs
     • pip install h5py
   • Ubuntu Libs
     • sudo apt install ncdu htop
   • Other DL Libs
     • sudo pip -Iv install dlib (if inside a virtualenv - AWS Deep Learning AMI)
       • Time Consuming (5mins + )
   • Check the used up size
     • df -h (Look for /dev/xvda1) --> (usually Used = 47G)
   • Check if all the above libraries are callable in a python interpreter

What-next?

ubuntu1604_nvidia_cuda.md

Ref

First Steps

• sudo apt update && sudo apt -y upgrade
• sudo apt install -y build-essential g++ gcc cmake
• Non-essential
  • sudo apt install -y htop ncdu
  • sudo apt install -y libboost-all-dev libopenblas-dev liblapack-dev
  • sudo apt install -y libgtk-3-dev libx11-dev

Nvidia

• Check Nvidia Graphics Card
  • lspci | grep -i nvidia
• Download Drivers
  • Option1 :
    • From here : http://www.nvidia.com/Download/index.aspx?lang=en-us
    • sudo dpkg -i nvidia-diag-driver-local-repo-ubuntu1604-384.111_1.0-1_amd64.deb
    • sudo apt update
    • sudo apt install cuda-drivers (#time consuming)
  • Option 2
    • wget http://us.download.nvidia.com/tesla/375.51/nvidia-driver-local-repo-ubuntu1604_375.51-1_amd64.deb
    • sudo dpkg -i nvidia-driver-local-repo-ubuntu1604_375.51-1_amd64.deb
    • sudo apt update
    • sudo apt install cuda-drivers # (Time consuming)
• Post Installation
  • cat /proc/driver/nvidia/version (may not work)
  • ls /usr/lib | grep nvidia-
  • nvidia-smi (to check if the driver was installed)

CUDA

• wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
  • sudo dpkg -i cuda-repo-ubuntu1604_9.0.176-1_amd64.deb
  • sudo apt update
• Download CuDNN from https://developer.nvidia.com/rdp/cudnn-download
  • sudo dpkg -i libcudnn7_7.1.1.5-1+cuda9.0_amd64.deb
  • sudo apt update
• Install CUDA9.0
  • sudo apt install cuda-9-0 (time consuming)
    • Add a symbolic link (on hold for now)
  • Check if CUDA is installed properly
    • cd /usr/local/cuda
    • cd 1_Utilities/deviceQuery
    • sudo make -j4
    • ./deviceQuery
    • If the above code fails then thry this
• Check if CuDNN is installed

  • function lib_installed() { /sbin/ldconfig -N -v $(sed 's/:/ /' <<< $LD_LIBRARY_PATH) 2>/dev/null | grep $1; }
    function check() { lib_installed $1 && echo "$1 is installed" || echo "ERROR: $1 is NOT installed"; }
    check libcudnn 
        libcudnn.so.7 -> libcudnn.so.7.0.4
    

• Add to path in ~/.bashrc:
  • export LD_LIBRARY_PATH=/usr/local/cuda/lib64/:/usr/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH
  • . ~/.bashrc

PYTHON LIBS

BASIC LIBS

• sudo apt install python-pip python3-pip python3-dev

NEURAL NET LIBS

• sudo pip3 install --upgrade tensorflow-gpu
• sudo pip3 install keras

OTHER NEURAL NET LIBS

• sudo pip -Iv install dlib (if inside a virtualenv - AWS Deep Learning AMI)

OTHER PYTHON LIBS

• sudo pip3 install jupyter
  • sudo pip3 install jupyter_contrib_nbextensions
  • jupyter contrib nbextension install --user
  • jupyter notebook --generate-config
  • jupyter kernelspec list : check is kernel.json file
  • jupyter notebook --ip=0.0.0.0
• sudo pip3 install jupyterlab
• Notes

  • 1. jupyter kernelspec list
       -> python2    /usr/local/share/jupyter/kernels/python2
    2. sudo vim /usr/local/share/jupyter/kernels/python2/kernel.json
       {
         "display_name": "Python 2",
         "language": "python",
         "argv": [
               "/usr/bin/python",
               "-m",
               "ipykernel_launcher",
               "-f",
           "{connection_file}"
         ]
       }
    

IMAGING LIBS

• sudo pip3 install opencv-python / sudo pip3 install opencv-contrib-python
• sudo pip3 install Pillow
• sudo pip3 install scikit-image
• sudo pip3 install shapely
• sudo pip3 install descartes
• sudo pip3 install matplotlib

STORAGE LIBS

• sudo pip3 install h5py

CLOUD-BASED LIBS

• pip3 install awscli

PYTHON-SERVER LIBS

• pip3 install flask

RANDOM LIBS

• sudo apt install python3-tk (to run matplotlib in command line)

COMMAND LINE UTILTIES

• sudo apt install ncdu (for files sizes within a folder)

RUNTIME HACKS

Nvidia Graphics Card (Processes)

• sudo fuser -v /dev/nvidia*

Nvidia Multi Graphics Senario

• os.environ["CUDA_VISIBLE_DEVICES"]="0"

Nvidia Monitoring Tools

• nvidia-smi
• watch -d -n 0.1 nvidia-smi
• nvidia-smi -l 1 | grep -i %
• nvidia-smi -r
• dstat tool
• montorix tool

Nvidia Boosting

• Boosting a Tesla K80

Running Flask Server with nohup

• Command : nohup python -u index.py &
• Logs : tail -f nohup.out