colllin

import torch
import psutil
import numpy as np

def log_profile(summaryWriter, step, scope='profile', cpu=True, mem=True, gpu=torch.cuda.is_available(), disk=['read_time', 'write_time'], network=False):
    if cpu:
        cpu_usage = np.array(psutil.cpu_percent(percpu=True))
        summaryWriter.add_scalars(f'{scope}/cpu/percent', {
            'min': cpu_usage.min(),
            'avg': cpu_usage.mean(),

colllin

In this article, I will share some of my experience on installing NVIDIA driver and CUDA on Linux OS. Here I mainly use Ubuntu as example. Comments for CentOS/Fedora are also provided as much as I can.

Table of Contents

• Install NVIDIA Graphics Driver via apt-get
• Install NVIDIA Graphics Driver via runfile
  • Remove Previous Installations (Important)
  • Download the Driver
  • Install Dependencies
  • Creat Blacklist for Nouveau Driver

• Stop lightdm/gdm/kdm

colllin

Prerequisites

• Write your training script so that it can be killed, and then automatically resumes from the beginning of the current epoch when restarted. (See train-example.py for an example training loop incorporating these recommendations.)
  • Save checkpoints at every epoch... (See utils.py for save_training_state helper function.)
    • model(s)
    • optimizer(s)
    • any hyperparameter schedules — I usually write the epoch number to a JSON file and compute the hyperparameter schedules as a function of the epoch number.
  • At the beginning of training, check for any saved training checkpoints and load all relevent info (models, optimizers, hyperparameter schedules). (See utils.py for load_training_state helper function.)
  • Consider using smaller epochs by limiting the number of batches pulled from your (shuffled) dataloader during each epoch.
• This will cause your trai

colllin

Development Setup

$ sudo add-apt-repository ppa:jonathonf/python-3.6
$ sudo apt update
$ sudo apt install python3.6 python3.6-dev
$ wget https://bootstrap.pypa.io/get-pip.py
$ python3.6 get-pip.py
$ rm get-pip.py
$ sudo pip3.6 install pipenv

colllin

# xvdg1 and xvdh1 are 2 attached volumes; md0 will be the (virtual) raid volume
mdadm -C /dev/md0 -l raid0 -c 64 -n 2 /dev/xvdg1 /dev/xvdh1 
mdadm -E /dev/xvd[g-h]1
mdadm --detail /dev/md0
mkfs.ext4 /dev/md0
df -h
mount /dev/md0 /mnt/

colllin

1. Install NVIDIA drivers

   1. Find NVIDIA driver download link for your system at http://www.nvidia.com/Download/index.aspx?lang=en-us

   2. wget -P ~/Downloads/ http://us.download.nvidia.com/tesla/390.46/NVIDIA-Linux-x86_64-390.46.run

   3. sudo rm /etc/X11/xorg.conf # It's ok if this doesn't exist

   4. NVIDIA will clash with the nouveau driver so deactivate it:

      $ sudo vim /etc/modprobe.d/blacklist-nouveau.conf
      

colllin

# Based on https://github.com/pytorch/pytorch/pull/3740
import torch
import math

class AdamW(torch.optim.Optimizer):
    """Implements AdamW algorithm.

    It has been proposed in `Fixing Weight Decay Regularization in Adam`_.

    Arguments:

colllin

These notes mostly follow a tutorial on installing NVIDIA and CUDA to avoid collisions with X servers.

https://medium.com/@DJVJallday/a-how-to-on-deep-reinforcement-learning-setup-aws-with-keras-tensorflow-openai-gym-and-jupyter-88bc0cc67e02

Backup Links:

• https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html
• forgets to discuss the --overide --no-opengl-libs when installing cuda: https://gist.github.com/wangruohui/df039f0dc434d6486f5d4d098aa52d07

Open questions:

colllin

# From https://gist.github.com/colllin/c02319fe3202470cc4d0a0b73cdbd1a6

#!/usr/bin/env python
###############################################################################
# $Id$
#
# Project:  GDAL2Tiles, Google Summer of Code 2007 & 2008
#           Global Map Tiles Classes
# Purpose:  Convert a raster into TMS tiles, create KML SuperOverlay EPSG:4326,
#           generate a simple HTML viewers based on Google Maps and OpenLayers

colllin

"""
Dynamic Routing Between Capsules
https://arxiv.org/abs/1710.09829
"""

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F
import torchvision.transforms as transforms