Skip to content

Instantly share code, notes, and snippets.

@Arbz583
Forked from Ekrekr/local.sh
Created April 23, 2024 18:23
Show Gist options
  • Save Arbz583/87a958fb9030a72e12e0aa8403ea6662 to your computer and use it in GitHub Desktop.
Save Arbz583/87a958fb9030a72e12e0aa8403ea6662 to your computer and use it in GitHub Desktop.
Run jupyter notebooks remotely using a custom python environment.
#!/bin/bash
# local.sh - access a remote running jupyter notebooks, for example on a
# cluster, locally. These steps can be run from any directory on your local
# machine.
# Create a tunnel for notebook communication. The user and server credentials
# should be the same as you used to log in to the server for the `remote.sh`
# script.
ssh -N -f -L localhost:8888:localhost:8890 <user>@<server>
# Open the notebook in browser. This step only works on a mac, so if you are
# using linux just open the web page in browser.
open "http://localhost:8888"
#!/bin/bash
# remote.sh - run jupyter notebooks remotely, for example on a cluster. These
# steps should be run from your project directory.
# Set up virtual environment. If python3 isn't available then use
# `ls /usr/bin/python*` to see available installations. If none are shown, use
# `module avail` to list modules, and you'll have to include
# `source <python version>` in a job submission script.
python3 -m venv venv
# Sourcing the environment should override your current python3 command.
# Alternatively the correct python can be directly accessed via
# `venv/bin/python3`.
source venv/bin/activate
# Optional: Install additional modules via a script. Alternatively `pip install
# --requirement <requirements.txt>` is a good option if you have a requirements
# file. This can be generated with `pip freeze > requirements.txt`
# venv/bin/pip install <module1> <module2> <...>
# Set up a local jupyter notebooks module in the project.
python3 -m ipykernel install --name venv --display-name venv --user
# Launch the notebook. Port 8890 is used so as not to interfere with any locally
# running notebooks.
jupyter notebook --no-browser --port=8890
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment