When developing, you generally want to have a reproducible environment, for each project. Maybe you project is currently using numpy as 1.19 version, but what if numpy v2 is so cool that despite breaking changes you want to use it in your new projects? If you have numpy installed globally, it will be a headache: if you step up, every old projects will be un-usable (which is inconvenient); if you never step up anything, you will be stuck in 2020 forever (which is sad).
Hence, it is best practice to keep things tidy, and to have specific environment to work with for each of your projects, containing all dependencies this project needs in order to run properly. In Python, those seem to be called "virtual environments" (seriously, across programming communities semantics can be tricky: what exactly is a module? a package? a dependency? an environment? an installation...?).
When developing with JavaScript, things are pretty straightforward thanks to npm. As a newcomer in Python, I wanted to avoid this kind of mess, but things seem a bit more complex and I was struggling to make my global JupyterLab installation work with a local python environment/installation (I do not want, for each of my projects, to re-install my IDE (integrated development... environment! boastful name for those fancy text editors)). Hence, this little guide.
Very much like the node_modules/ subfolder that npm handles, I want a envs/ subfolder at the root of my project directory (it is the ./envs below but you can change it to whatever name please your ear). Why? So I don't lose sight of what project is based on what: I will have everything in the same directory, so I won't forget that 1) I'm using a specific environment, 2) I installed this and that here, with this and that versions. (This can also have some downsides, see here.)
Hereafter, I will make the strong assumption that you already are using conda, with Anaconda or Miniconda (the latter untested, see a comparison). One advantage of conda is – according to what I read – its versatility. It works with dependencies written in a variety of languages. I also use Windows (shame!).
We will need some stuff installed globally. If they are not installed on your machine already... well, install them!
- JupyterLab is a cool IDE, but if you are here you probably already know that.
conda install jupyterlab- nb_conda_kernels will make local conda "kernels" pop up in your globally installed JupyterLab.
conda install nb_conda_kernelsYou will also have to initialize conda on your system terminal.
conda init powershell
# + Avoid auto conda launch for each terminal instances.
conda config --set auto_activate_base falseOnce things are set up globally, you can follow these steps below to create and use a local environment.
# First you have to go to the root directory of your project.
cd path/to/your/project
# Create environment in a local directory.
conda create --prefix ./envs
# Activate environment from the local directory.
conda activate ./envs
# You are now in the conda local environment.
# You can now install any package you want in the environment, e.g.:
conda install numpy
# Set up a kernel pointing to the local environment.
conda install -c anaconda ipykernel
python -m ipykernel install --prefix=./envs --name 'your_project_name'
# Launch JupyterLab.
jupyter lab
# Close JupyterLab (these are keystrokes, not a command line).
[ctrl + c]
# Deactivate (go out from) your conda local environment.
conda deactivateWithin JupyterLab, the kernel associated with the project environment should be named your_project_name [conda env:envs]. Beware: if we make the same for multiple projects, their respective environments will be named envs-2, envs-3... That's why we put 'your_project_name' as the kernel name, otherwise it would only be named Python [conda env:envs] which can become confusing.
Once set up, next time you will be able to launch your project in JupyterLab with:
conda activate ./envs
jupyter labWhen JupyterLab is launched from the environment, the inside JupyterLab terminal should also be in the Conda environment by default, so you will be able to install from it any package to your virtual environment.
Note: If you are using git, you probably also want to ignore the envs/ dependencies folder. So, add envs/ to your .gitignore file (if you do not already have such file, just create it at the root of your project directory as a blank slate and add envs/ as a single line).
Other command lines that could be of some use:
# Install packages, from a specific version,
# directly when creating the environment.
conda create --prefix ./envs matplotlib=3.1 numpy=1.16
# List kernels within the current environment.
jupyter kernelspec list
# Remove a kernel.
jupyter kernelspec uninstall your_kernel_name-
dev.to/bowmanjd/python-tools-for-managing-virtual-environments-3bko#conda
-
docs.conda.io/.../manage-environments#specifying-a-location-for-an-environment
| Software | Version |
|---|---|
| Windows 10 | 1909, 10.0.18362.1171 |
| Windows PowerShell | 5.1.18362.1171 |
| Python | 3.8.5 |
| conda | 4.9.2 |
| jupyter core | 4.6.3 |
| jupyter client | 6.1.7 |
| jupyter lab | 3.0.5 |
| ipython | 7.18.1 |
| ipykernel (local) | 7.18.1 |