Google Colab Hacks

check_gpu_usage.py

# memory footprint support libraries/code
# ! ln -sf /opt/bin/nvidia-smi /usr/bin/nvidia-smi
# ! pip install humanize
# ! pip install jupyter_contrib_nbextensions
# ! jupyter contrib nbextension install --user
# ! jupyter nbextension enable codefolding/main
# ! jupyter nbextension enable hinterland/hinterland


import psutil
import humanize
import os
import GPUtil as GPU
GPUs = GPU.getGPUs()
gpu = GPUs[0]
process = psutil.Process(os.getpid())
print("Gen RAM Free: " + humanize.naturalsize( psutil.virtual_memory().available ), " | Proc size: " + humanize.naturalsize( process.memory_info().rss))
print("GPU RAM Free: {0:.0f}MB | Used: {1:.0f}MB | Util {2:3.0f}% | Total {3:.0f}MB".format(gpu.memoryFree, gpu.memoryUsed, gpu.memoryUtil*100, gpu.memoryTotal))

dir_structure.md

Directory Structure

1. src/
   • this folder contains all the .py code files such as
     • nets.py (containts classes like VGG(), YOLO(), ResNet() etc)
     • train.py (contains classes like VGGTrain(), YOLOTrain() etc)
     • test.py (~)
     • dataloader.py (contains classes like PASCALVOC(), MNIST() etc)
2. demo/
   • this folder contains subfolders such as demo1/, demo2/ etc
   • inside each folder one can find .ipynb notebook with different experiments
     • within each of these jupyter notebook, you can the classes and their functions defined in the src/*.py files

Notes

• make sure to include a __init__.py empty file in each directory

downloading.py

# %%time
# ! git clone --branch feature/prerak2 https://github.com/prerakmody/CS4180-DL
# ! wget -P /content/CS4180-DL/data/dataset https://pjreddie.com/media/files/VOCtrainval_11-May-2012.tar
# ! tar xf /content/CS4180-DL/data/dataset/VOCtrainval_11-May-2012.tar --directory /content/CS4180-DL/data/dataset
# ! wget -P /content/CS4180-DL/data/dataset https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar & tar xf VOCtrainval_06-Nov-2007.tar   
# ! tar xf /content/CS4180-DL/data/dataset/VOCtrainval_06-Nov-2007.tar --directory /content/CS4180-DL/data/dataset
# ! wget -P /content/CS4180-DL/data/dataset https://pjreddie.com/media/files/VOCtest_06-Nov-2007.tar & tar xf VOCtest_06-Nov-2007.tar
# ! tar xf /content/CS4180-DL/data/dataset/VOCtest_06-Nov-2007.tar --directory /content/CS4180-DL/data/dataset
# ! mkdir /content/CS4180-DL/data/weights

"""
 - Please upload the model weights to "CS4180-DL/data/weights"
"""

! ls -l --block-size=M

could also add "! pip install GPUtil" as it is needed package.