mehdirezaie

Jupyer Notebook

Jupyter Notebook has become an essential element of exploratory data analysis and data visualization pipelines. In this note, I will give the instructions on how to start a jupyter kernel remotely and connect to it.

1. On the server, install Jupyter Notebook
2. On the server, initialize a kernel using an arbitrary port number (e.g., 1234) jupyter notebook --no-browser --ip=localhost --port=1234. Copy the token number from the output log. Note you can use any port number.
3. On the local, execute ssh -N -f -L localhost:8888:localhost:1234 user@server where you need to replace user with your username and server with your server. This command will initiate a channel from server's localhost:1234 to local's localhost:8888. Again, you can use any port number. You just need to be consistent.
4. Go to localhost:8888 in your local browser, you may need to enter the token number which you have copied in step 2.

In case you forget the Token number, execute the following command on the

mehdirezaie

Profiling in Python

Main reference: https://github.com/csurfer/pyheatmagic
See also https://stackoverflow.com/a/52461806/9746916

Step 1. Install py-heat-magic

%pip install py-heat-magic # run this only once for installation

mehdirezaie

Three axes visualizations

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

mehdirezaie

Jobs Array

Main Reference: https://docs.nersc.gov/jobs/examples/#job-arrays
"Job arrays provides a mechanism for submitting and managing collections of similar jobs quickly and easily." You will create a single script (e.g., job.sbatch) which uses the variable SLURM_ARRAY_TASK_ID to point at the correct files:

#!/bin/bash
#SBATCH -q debug
#SBATCH -o job_array_test%j.out
#SBATCH -n 1
#SBATCH --time 00:02:00
#SBATCH -C haswell

mehdirezaie

Pi Calculation in C vs Python

This is another example which we show how we can wrap a function in C to gain more performance. We use the integral of $\int_{0}^{1} \frac{4}{1+x^{2}} dx$ to calculate $\pi$.

This is the C code:

/*
    Code to compute the number Pi = 3.14

    Integral 4/(1+x^2) from 0 to 1 is 3.14

mehdirezaie

#!/bin/bash
# This has to be run from master
git checkout master

# Update our list of remotes
git fetch
git remote prune origin

# Remove local fully merged branches
git branch --merged master | grep -v 'master$' | grep -v 'dev$' | xargs git branch -d

mehdirezaie

Colormaps and Contours

This example illustrates how to visualize two maps with colorbars and contours.

x = np.linspace(-10, 10, num=100)
y = np.linspace(-10, 10, num=100)
X, Y = np.meshgrid(x, y)

Z1 = X*X + Y*Y
Z2 = abs(X) + abs(Y)

mehdirezaie

Colorblind friendly visualization

import matplotlib.pyplot as plt
import numpy as np
from cycler import cycler


colors = ['#377eb8', '#ff7f00', '#4daf4a',
 '#f781bf', '#a65628', '#984ea3',

mehdirezaie

Publication Quality with Python

import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt

mpl.rcParams['font.family'] = 'Liberation Sans Narrow'
plt.rcParams['font.size'] = 18

mehdirezaie

For best VIM settings (in your ~/.vimrc):

filetype plugin indent on
syntax on
set number
set tabstop     =4
set softtabstop =4
set shiftwidth  =4
set expandtab