- Reproduction of core content of ATLAS Talk Topic: Preferred way of creating a Jupyter kernel for Python virtual environment (ATLAS Internal, sorry)
- Corresponding YouTube explainer video: Creating an IPython kernel for your venv on UChicago AF Jupyter Lab (no captions, sorry)
For robustness and reproduciblity, I would like to use a Python virtual environment when working with ml_platform:latest
containers (easy enough) but I would also like to have IPython kernels that I make in that virtual environment be recognized by Jupyter Lab's launcher. I believe that the easiest way to do this is to just use the --user
flag when creating an IPython kernel, but I wanted to know if there is an alternative / create this as an example Issue for others to reference.
Here's an example of what I'm doing in a ml_platform:latest
container:
$ mkdir example
$ cd example
$ python3 --version
Python 3.8.10
$ python3 -m venv example-venv && . example-venv/bin/activate
$ python -m pip --quiet install --upgrade pip setuptools wheel
$ python -m pip list
Package Version
------------- -------
pip 23.0
pkg_resources 0.0.0
setuptools 67.1.0
wheel 0.38.4
$ python -m pip --quiet install tensorflow ipykernel
$ python -m ipykernel install --help
usage: ipython-kernel-install [-h] [--user] [--name NAME] [--display-name DISPLAY_NAME] [--profile PROFILE] [--prefix PREFIX] [--sys-prefix]
[--env ENV VALUE]
Install the IPython kernel spec.
optional arguments:
-h, --help show this help message and exit
--user Install for the current user instead of system-wide
--name NAME Specify a name for the kernelspec. This is needed to have multiple IPython kernels at the same time.
--display-name DISPLAY_NAME
Specify the display name for the kernelspec. This is helpful when you have multiple IPython kernels.
--profile PROFILE Specify an IPython profile to load. This can be used to create custom versions of the kernel.
--prefix PREFIX Specify an install prefix for the kernelspec. This is needed to install into a non-default location, such as a conda/virtual-env.
--sys-prefix Install to Python's sys.prefix. Shorthand for --prefix='/home/feickert/example/example-venv'. For use in conda/virtual-envs.
--env ENV VALUE Set environment variables for the kernel.
$ python -m ipykernel install --user --name="example-venv" --display-name="Example venv kernel"
Installed kernelspec example-venv in /home/feickert/.local/share/jupyter/kernels/example-venv
This then allows for viewing and using the kernel from Jupyter Lab's launcher. In the below screenshots you can see this as well as verify that the virtual environment that the kernel was created in is getting picked up and used. 👍