Skip to content

Instantly share code, notes, and snippets.

@QuanchaoSong

QuanchaoSong/Using 'conda-forge' channel only.md

Last active November 28, 2023 11:46
Show Gist options
  • Save QuanchaoSong/0bbf4ee101e4c3057a6a72ec1403dbdd to your computer and use it in GitHub Desktop.
Save QuanchaoSong/0bbf4ee101e4c3057a6a72ec1403dbdd to your computer and use it in GitHub Desktop.
Download ZIP
GSI-Internship-QuanchaoSong
Raw
Using 'conda-forge' channel only.md

Using 'conda-forge' channel only

Date: 2023-02-21

Problem

For some reason, we can't use anaconda or miniconda in a normal way anymore, otherwise our purse will get hurt. Normally, we download python packages from anaconda's defaults channel, which is maintained by anaconda's official team. Now we need to stop downloading from the "defaults" channel. Instead, we have to turn to "conda-forge" channel.

There are 3 ways to handle this issue, which will be expounded below. The first way is specially suggested.

No.1 way: to use mambaforge (Suggested)

Introduction to mambaforge

Before introducing mambaforge, we have to know something about miniforge.

miniforge is a variation for miniconda specifit to "conda-forge" channel. Everything is the same as miniconda when we use miniforge, the only difference is that it downloads python packages from the "conda-forge" channel by default. And that's what we want, indeed.

Then what's the difference between mambaforge and miniforge? And why we suggest it more than miniforge?

Actually, when we use these two tools, there is almost no difference between them. Same as miniforge, mambaforge also uses conda-forge as the default channel to download packages. Every command of mambaforge remains consistent with that of condaforge. The tiny difference is to simply replace conda with mamba, when we try to do something with the python packages.

There are some examples to prove it in the following.

E.g., to install a package:

### for installing a package, in `condaforge`, we have:
conda install scipy
### in `mambaforge`, we have:
mamba install scipy

E.g., to create a virtual environment:

### create & enter a new virtual enviroment in `condaforge`:
conda create -n mywork python=3.10 --yes --no-default-packages
conda activate mywork
### while in `mambaforge`:
mamba create -n mywork python=3.10 --yes --no-default-packages
mamba activate mywork

Here are a full list of shared commands between mamba and conda:

### supported by both `mamba` & `conda`:
install, create, list, search, run, info, clean, activate and deactivate

The essential difference lies here: mambaforge uses C/C++ language to implement some functions within the package manager program. For functions such as packages' downloading & environment resolving, those C/C++ codes bring the package manager with higher efficiency and versatility. Basically, we can say that, mambaforge can save much of your time while installing python packages, for example, it supports multi-thread downloading therefore the downloading procedure goes faster, and its environment resolving is much faster than conda, etc.

Preparation

If you have already installed anaconda/miniconda, to avoid potential confilicts with mambaconda, it's better to uninstall it.

Here are several steps to uninstall it.

  1. Remove anaconda/miniconda related folder:

    rm -rf ~/anaconda/
### or
rm -rf ~/miniconda/

rm -rf ~/.conda/

  2. Remove configuration file:

    rm -f ~/.condarc

  3. Delete some sentences within .bashrc or .bash_profile like the following (if there are):

    # >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/albus/Downloads/yes/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/albus/Downloads/yes/etc/profile.d/conda.sh" ]; then
        . "/home/albus/Downloads/yes/etc/profile.d/conda.sh"
    else
        export PATH="/home/albus/Downloads/yes/bin:$PATH"
    fi
fi
unset __conda_setup

if [ -f "/home/albus/Downloads/yes/etc/profile.d/mamba.sh" ]; then
    . "/home/albus/Downloads/yes/etc/profile.d/mamba.sh"
fi
# <<< conda initialize <<<

    Then run command:

    source ~/.bashrc
### or
source ~/.bash_profile

    or just close your terminal & restart it.

Until now, anaconda/miniconda is totally removed from your computer.

Installation of mambaforge

  1. Go to this page, choose a suitable version to download. In my case, I choose Mambaforge-Linux-x86_64. You can download it by whatever tool you wish, e.g. using wget. Open terminal, run following commands:)

    cd ~/Downloads/    ### or any other common folder
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh

### then install it
sh Mambaforge-Linux-x86_64.sh

  2. After installation, close your terminal and restart it to make sure the bash environment is set up correctly. You should see "(base)" in front of each prompt input line.

    But in some cases, if you run any conda/mamba command like conda list/mamba list, the terminal pops an error like "conda/mamba: command not found", then your installation is not complete. For this error, we can solve it by re-init our mamba/conda environment as the following:

    # In the terminal
# Firstly let's re-init our `mamba/conda` environment, by which some codes will be writen in `~/.bashrc` file.
~/mambaforge/bin/mamba init

# Then we refresh our terminial enviroment
source ~/.bashrc

    The above two simple bash commands should be enough to complete the installation now. However, if it still reports an error like conda/mamba: command not found when you type conda/mamba command in the terminal, then we can create soft links to the conda and mamba commands.

    ln -s ~/mambaforge/bin/conda /usr/bin/conda
ln -s ~/mambaforge/bin/mamba /usr/bin/mamba
### or
ln -s ~/mambaforge/bin/conda /usr/local/bin/conda
ln -s ~/mambaforge/bin/mamba /usr/local/bin/mamba

    Now if you type conda or mamba again in the terminal, it should work properly.

Experiment on mambaforge

Since installation is finished, we can check it a bit.

Firstly, let's see if the packages are from conda-forge channel:

### make the `conda list` command show the `Channel` column
conda config --set show_channel_urls yes

conda list
### output should look like the following:
#   Name                    Version                   Build    Channel
# _libgcc_mutex             0.1                 conda_forge    conda-forge
# _openmp_mutex             4.5                       2_gnu    conda-forge
# appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
# ...
# ...

As we can see, now the packages are all from conda-forge channel.

Then, let's install some popular python packages and test it:

mamba install scipy pandas
python -c "import scipy"
### Nothing should be printed here if success.

Here is another example of installing CERN-ROOT package, which is used very often in Physics:

mamba install root
python -c 'from ROOT import TCanvas'

If nothing is printed here, it means the packages are installed without any error, then all is fine.

Now everything works well with mambaforge.

No.2 way: to use miniforge

As mentioned before, mambaforge and miniforge are quite similar package tools, between which the major difference is that mambaforge is faster for the installation of python packages. However, if you insist on using miniforge, it's still ok.

The installation and setup for miniforge is actually the same as that formambaforge.

  1. Go to this page, choose a suitable version to download. In my case, I choose Miniforge3-Linux-x86_64.

    Open terminal and run:

    cd ~/Downloads ### or any other common folder
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh

### then install it
sh Miniforge3-Linux-x86_64.sh

  2. In some cases, if you run any conda command like conda list, the terminal pops an error like "conda: command not found", then your installation is not complete. For this error, we need to create a soft link to the conda command:

    ln -s ~/miniforge3/bin/conda /usr/bin/conda
### or
ln -s ~/miniforge3/bin/conda /usr/local/bin/conda

    Now everything should be fine with miniforge.

  3. Now test miniforge a bit.

    Firstly, configure conda to make conda list display Channel column in its output.

    conda config --set show_channel_urls yes

    Then install a random python package:

    conda install scipy

    Now run conda list, you should see that all the installed packages are from the "conda-forge" channel according to the values under the Channel column of the output.

No.3 way: to change configuration of miniconda

This way is used only when you are very obsessed with the original miniconda stuff.

Before everything below, please firstly make sure that you are using a network that is accessible to repo.anaconda.com!

1.Install miniconda:

The difference between anaconda and miniconda is just the amount of pre-installed packages. So you can safely start with miniconda. If you haven't installed miniconda, firstly install it (requires non-blocked network access).

  1. Go to webpage: https://docs.conda.io/en/latest/miniconda.html, choose the correct version to download. Here my choice is: Miniconda3 macOS Intel x86 64-bit bash

  2. Install the downloaded package, according to the instructions. e.g. by doing

sh Miniconda3-latest-MacOSX-x86_64.sh

### or(on Linux)
sh Miniconda3-latest-Linux-aarch64.sh
  1. In some cases the soft link for conda command is not created properly, then you have to create it like:
ln -s ~/miniconda3/bin/conda /usr/bin/conda
### or
ln -s ~/miniconda3/bin/conda /usr/local/bin/conda
  1. Now enter conda -V, to see if a correct version message is printed. In my case, it prints: conda 23.1.0, which means miniconda is installed correctly. Also, your terminal may show a "(base)" prefix before each prompt input, which means it has entered the virtual environment that is created by miniconda automatically.
2.Set up

By default, either anaconda or miniconda installation file already contain some basic python packages from defaults channel within itself. Once you download the installation file and finish the installation, some local python packages will be extracted & installed along the procedures that you install miniconda. It means, even though you turn off your network, these python packages from defaults channel are still there. So we have to do something regarding this issue.

Firstly, restart your terminal, and please enter the following line:

conda config --set show_channel_urls yes

And look at the last column "Channel" of the output of the following command:

conda list

Here defaults means anaconda.com, and conda-forge means anaconda.org. The latter is what we want all packages to be. If we don't do anything, all packages will get installed or updated from the defaults channel. This is something that should be avoided. So we need to remove the defaults channel first and then update using conda-forge channel.

Therefore, we need to change the configuration of miniconda, to make it avoid downloading packages from the "defaults" channel.

To do this, run the following commands one by one in your terminal:

# now it should include `defaults` channel
conda config --show-sources
### results should be like
# channels:
#   - defaults
# show_channel_urls: true

# Add new channel
conda config --add channels conda-forge

# delete the "defaults" channel
conda config --remove channels 'defaults'

# now it should only include `conda-forge` channel
conda config --show-sources
### results should be like
# channels:
#   - conda-forge
# show_channel_urls: true

Finally & very importantly, update all packages:

conda update --all

Now you can check the results using conda list to see if all pacakges are showing conda-forge as channel.

From now on you can install any python packages you like:

conda install scipy

and miniconda will download scipy from the "conda-forge" channel.

By the way, you can create a new virtual environment:

conda create -n mywork python=3.10 --yes --no-default-packages
conda activate mywork
conda install scipy pandas

All packages will be downloaded from "conda-forge" channel.

Please be noted that, the 3rd way is NOT able to change the channel for one special package to conda-forge, which is conda itself. This means in the future, when you update all packages, the package conda will still be downloaded from defaults channel; and when you use a network that blocks defaults channel, update for the package conda will fail. However, we have figured out a small trick to solve this, just run following commands:

### roll package `conda` back to a slightly older version, let's say: 22.9.0
conda install conda=22.9.0
conda update conda
conda list

Now all packages including conda should be from the channel conda-forge.

Enjoy using conda!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment