Skip to content

Instantly share code, notes, and snippets.

@ptooley
Last active November 28, 2017 16:41
Show Gist options
  • Save ptooley/a36f3a48a9b4cd4db341574c5d7ea5dd to your computer and use it in GitHub Desktop.
Save ptooley/a36f3a48a9b4cd4db341574c5d7ea5dd to your computer and use it in GitHub Desktop.
custom kernel environments for jupyter-sharc.shef.ac.uk

Conda environments

By default the conda environment provided for sharc-jupyter is read only. To be able to have a configurable environment it must be created in our homedir. Once we have created our conda environment we can then register it as an ipython kernel, and it will become available in the kernel list for sharc-jupyter instances.

Creating the environment

In this example I will create an intel python environment, which is good choice for hpc use as it is built using intel's optimised libraries. But the same approach will work for any other conda environment.

New environments are created via the linux shell, which we can use the ipython terminal emulator to access: From the files tab, choose New -> Terminal. Alternatively use qrshx to get an interactive sharc session and load the conda module. Then, use the following commands to create a new environment.

%(optional) add intel channel to get the intel python packages
conda config --add channels intel

%create a new environment based on e.g intelpython3_core template
conda create -n myenvironment intelpython3_core jupyter_client

This will give you a minimal ipython kernel for jupyter notebook. Now from any running sharc-jupyter notebook, going to 'Kernel -> Change Kernel' in the menu will allow you to switch to the newly created environment. In addition, any servers spawed after you create the kernel will allow you to create new notebooks using this kernel as the default.

To install additional packages, you can use the jupyter terminal to do so.

%enter new environment
source activate myenvironment

%install other desired packages
conda install mpi4py pyfftw ....

R environments

R environments may be similarly created using the following:

conda create -n R-env python=3.6 r-irkernel jupyter_client

In my case I also had to install libiconv as a missing dependency

source activate R-env
conda install libiconv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment