Custom conda environment on Azure ML Compute Instance

Conda_env_on_azureml_compute_instance.md

Pre-reqs

From the terminal running on the AML compute instance, ensure that conda is accessible in your current shell session.

which conda
#-> should return a path to a conda executable

If conda is not found, then verify that the executable is present where expected

ls /anaconda/bin/conda
#-> should return /anaconda/bin/conda

Then run

/anaconda/bin/conda init

#-> this should return a notics that your bashrc has been modified.

Close and re-open the terminal. It should now launch with the base environment activated, indicated by (base) in front of the prompt.

Creating a custom environment for use on an Azure ML compute instance

Create an environment definition and store it in a yml file, e.g environment.yml. Include ipython and ipykernel as dependencies.

IMPORTANT: to use the anaconda channels included in defaults in commercial contexts, a licence is required: Terms of service

File: environment.yml: ⚠️ This includes v1 azureml-core, update for v2

name: testenv
channels:
- conda-forge
- defaults
dependencies:
- python=3.8
- numpy
- [any other dependencies available through conda / conda-forge]
- pip
- pip:
  - azureml-core
  - ipython
  - ipykernel
  - [any other dependencies]

In the terminal for the Azure ML compute instance:

# ensure no environment is active other than base
conda deactivate

# verify that the base environment is active (starred):
conda env list

# create environment from file
conda env create -f environment.yml

# activate environment (name `testenv` was configured in the file)
conda activate testenv

# make the kernel available to juptyer:
ipython kernel install --user --name testenv --display-name "My Custom Name"

Close any existing notebooks and reopen them in the integrated notebook interface, or navigate to the Jupyter interface from the compute instance page and open the notebook from there.

The new environment should be available in the kernel dropdown, named My Custom Name

Verifying the active kernel from within the notebook

Verify the environment in use by the notebook with:

import sys
print(sys.version)
print(sys.executable)

Editing or changing registered kernels

The registered kernels are maintained in the following location - they can be edited or deleted here:

from a terminal:

ls ~/.local/share/jupyter/kernels/

See also

The official docs show a simplified route where the environment is not defined in a file - install any required packages on the fly with the %conda magic command from within the notebook once the new environment is active:

• https://docs.microsoft.com/en-us/azure/machine-learning/how-to-access-terminal#add-new-kernels

Note: recommend specifying the python version in the creation step: conda create -n newenv python=3.x