Execution of some code on CSCS

cscs-experiments.md

The first time

Add this line to your .bashrc file.

export PATH=$PATH:~/.local/bin

Then, load the following modules.

module load daint-gpu
module load cray-python
module load TensorFlow

To use pytorch, just replace the last line with module load PyTorch.

You should be ready to install virtualenv

pip install virtualenv --user

Now you should be able to create virtual environnements

virtualenv --system-site-packages -p python3 default
source default/bin/activate 
pip install matplotlib jupyter sklearn ipython

The next times

Load the modules and then load you virtual environnement

module load daint-gpu
module load cray-python
module load TensorFlow

source default/bin/activate

Require the allocation of a node to experiment with it

Require the node

This command require 1 node with GPU for 360 minutes

salloc -N 1 -C gpu -t 360 -A sd04

One the node is allocated, you can launch a bash using

srun --pty bash

Then you can work interactively...

Launch experiments

When you are done playing with a node, you can launch serious experiments using sbatch.

1. Make a script to launch your experiment. Here is an example:

#!/bin/bash -l
#SBATCH --time=23:59:00
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --constraint=gpu
#SBATCH --output=wgan64-test-%j.log
#SBATCH --error=wgan64-test-%j.log
#SBATCH --account=sd01

module load daint-gpu
module load cray-python
module load TensorFlow

source $HOME/default/bin/activate

cd $SCRATCH/nati-gpu/upscale_gan_testing/nati_experiments/
srun python WGAN64-test.py

2. Launch the experiment using sbatch

sbatch script.sh

Using tensorboard

Set up port forwarding or a proxy and run...

module load daint-gpu
module load cray-python
module load TensorFlow

tensorboard --logdir .

Jupyter notebook

Check: https://user.cscs.ch/tools/interactive/jupyterlab/