Skip to content

Instantly share code, notes, and snippets.

@xkortex
Last active September 15, 2020 20:49
Show Gist options
  • Save xkortex/5ae49d7e6e969405bd2c3152a949c1f1 to your computer and use it in GitHub Desktop.
Save xkortex/5ae49d7e6e969405bd2c3152a949c1f1 to your computer and use it in GitHub Desktop.
improved conda SHELL for docker RUN
#!/bin/bash
## Entrypoint for running docker ENTRYPOINT with conda env
## Enable by adding:
## COPY conda_entry.sh /conda_entry.sh
## ENTRYPOINT ["/conda_entry.sh"]
##
## Optionally, set the following env to select a conda env to run in
## ENV CONDA_DEFAULT_ENV=foo
## You may also want to add something like
## RUN conda init bash && echo 'conda activate "${CONDA_DEFAULT_ENV:-base}"' >> ~/.bashrc
## to drop into a default env when `docker exec -it $IMAGE bash`
## Docker shells by default run as nonlogin, noninteractive
## More references:
## https://pythonspeed.com/articles/activate-conda-dockerfile/
## https://stackoverflow.com/questions/56510575/activate-and-switch-anaconda-environment-in-dockerfile-during-build
## https://stackoverflow.com/questions/37945759/condas-source-activate-virtualenv-does-not-work-within-dockerfile/62803490#62803490
## It is insufficient to run `conda init bash` in a dockerfile, and then `source $HOME/.bashrc` in the entry script.
## This is mostly because the `RUN` directives are noninteractive, non-login shells, meaning `.bashrc` is never
## sourced, and `RUN source` does not behave the way one might naively think it should
## However, by taking the `conda shell.bash hook` directly, we end up with a conda-tized
## RUN directive!
## The conda shell hook placed in `.bashrc` will reset our
## env to "base" on shell-ing into the container. If you want to start in a custom end,
## cache the value because the shell hook step will remove it
_CONDA_DEFAULT_ENV="${CONDA_DEFAULT_ENV:-base}"
__conda_setup="$('/opt/conda/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
eval "$__conda_setup"
unset __conda_setup
## Restore our "indended" default env
conda activate "${_CONDA_DEFAULT_ENV}"
## This just logs the output to stderr for debugging.
#>&2 echo "ENTRYPOINT: CONDA_DEFAULT_ENV=${CONDA_DEFAULT_ENV}"
exec "${@}"
#!/bin/bash
## in your Dockerfile
# COPY conda_run.sh /conda_run.sh
# RUN chmod +x /conda_run.sh
# SHELL ["/conda_run.sh", "/bin/bash", "-c"]
## usage
## CONDA_DEFAULT_ENV=FOO command arg1 arg2
## or
## command arg1 arg2
__conda_setup="$('/opt/conda/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
eval "$__conda_setup"
unset __conda_setup
## The SHELL ["/run.sh"] command passes everything in the RUN stanza as a single string
## There may be a better way to unpack it
IFS=' ' read -ra ARGS <<< "${1}"
FIRST=${ARGS[@]::1}
## debugging, uncomment to see the args passed into this script
# echo ...
# echo "${1}"
# echo "${ARGS[@]}"
# echo ...
## parse the possible first argument for setting conda env
## This is not a "true environment variable", we just emulate the syntax for essentially syntactic sugar in Dockerfiles
if [[ "$( echo ${FIRST}| cut -c-18)" == "CONDA_DEFAULT_ENV=" ]]; then
_CONDA_DEFAULT_ENV=$(echo "${FIRST}" | cut -c19-)
EXEC_ARGS=$(echo ${ARGS[@]:1})
else
_CONDA_DEFAULT_ENV=base
EXEC_ARGS="$(echo ${ARGS[*]})"
fi
##logging, this is just for debugging, you can enable this to sanity check or see what is happening
#>&2 echo "ACTIVATING: ${_CONDA_DEFAULT_ENV}"
#>&2 echo "RUNNING: ${EXEC_ARGS}"
conda activate "${_CONDA_DEFAULT_ENV}"
/bin/bash -c "${EXEC_ARGS}"
## Conda with custom entrypoint from base ubuntu image
## Build with e.g. `docker build -t monoconda .`
## Run with `docker run --rm -it monoconda bash` to drop right into
## the environment `foo` !
FROM ubuntu:18.04
## Install things we need to install more things
RUN apt-get update -qq &&\
apt-get install -qq curl wget git &&\
apt-get install -qq --no-install-recommends \
libssl-dev \
software-properties-common \
&& rm -rf /var/lib/apt/lists/*
## Install miniconda
RUN wget -nv https://repo.anaconda.com/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh -O ~/miniconda.sh && \
/bin/bash ~/miniconda.sh -b -p /opt/conda && \
rm ~/miniconda.sh && \
/opt/conda/bin/conda clean -tipsy && \
ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh
## add conda to the path so we can execute it by name
ENV PATH=/opt/conda/bin:$PATH
## Create /entry.sh which will be our new shell entry point. This performs actions to configure the environment
## before starting a new shell (which inherits the env).
## The exec is important! This allows signals to pass
COPY conda_run.sh /conda_run.sh
COPY conda_entry.sh /conda_entry.sh
RUN chmod +x conda_run.sh && chmod +x /conda_entry.sh
## Tell the docker build process to use this for RUN.
## The default shell on Linux is ["/bin/sh", "-c"], and on Windows is ["cmd", "/S", "/C"]
SHELL ["/conda_run.sh"]
## Now, every following invocation of RUN will start with the entry script
RUN conda update -n base conda -y \
&& conda install -n base pip
## Create a dummy env
RUN conda create --name foo
## I added this variable such that I have the entry script activate a specific env
ENV CONDA_DEFAULT_ENV=foo
## This will get installed in the env 'foo' since it gets activated at the start of the RUN stanza
RUN conda install pip
## This shows our pseudo-env-variable-setting ability as seen in conda_run.sh line 18ish.
## btw xdoctest is fantastic, you should try it https://pypi.org/project/xdoctest/
RUN CONDA_DEFAULT_ENV=bar pip install xdoctest
RUN CONDA_DEFAULT_ENV=bar pip list | grep xdoctest
# this should be empty
RUN CONDA_DEFAULT_ENV=foo pip list | grep xdoctest
## Configure .bashrc to drop into a conda env and immediately activate our TARGET env
RUN conda init && echo 'conda activate "${CONDA_DEFAULT_ENV:-base}"' >> ~/.bashrc
ENTRYPOINT ["/conda_entry.sh"]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment