mlgill/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Automated Conda Setup

Michelle L. Gill

Updated: 2017/05/20
Regularly updated Gist located here.
This bash script will download and setup a Conda environment on Mac OS X or Linux using a specified Python version (2.7, 3.6) and packages. The Conda installation path and environment name can also be specified. All settings are input at the beginning of the script (see below).
If the Conda directory exists and can be determined to be a Conda installation, a new environment will be created. If the directory does exist but does not appear to be a Conda installation, the script will quit to avoid accidentally overwriting a file.
Running the Script

The script can be downloaded and run from any directory to which you have write access. To run the script, set the variables below. The Conda installation path (ENV_PATH) can be relative or absolute. Then make the script executable with chmod +x setup_conda.sh and run the script from the command line with ./setup_conda.sh.
# Set environment path
# $HOME can be used for a shortcut to your home directory
ENV_PATH="$HOME/miniconda"

# Set environment name
# NOTE: there's not an easy way to determine if an existing environment
# has the same name. Conda itself will throw an error during installation
# of second environment.
# ENV_NAME="scienv2"
ENV_NAME="scienv3"

# Set python version: 2.7, 3.6, etc
# PYTHON_VER=2.7
PYTHON_VER=3.6

# Set packages to be installed (do not list python itself)
# be sure to name/spell them exactly as Conda does or installation will fail
PACKAGES="psutil numpy scipy matplotlib seaborn pandas statsmodels jupyter ipython notebook nbconvert beautifulsoup4 html5lib lxml ipykernel requests flask scikit-learn nltk gensim pymongo"
Navigating the Conda Environment

To setup multiple environments, you can run the script more than once. Be sure to change the name and python version (if desired) for each environment:
$ conda env list

# conda environments:
#
scienv3               *  /Users/mlgill/miniconda/envs/scienv3
root                     /Users/mlgill/miniconda
Activating the Virtual Environment

The script will offer to modify your ~/.bashrc file to automatically load the new conda environment. However, you can change between environments at any time. For example:
$ source activate scienv3
You will see the following prompt (notice the virtual environment name now appears in front of the $):
(scienv3) $ 
Voila!  You can now run the Jupyter notebook.
(scienv3) $ jupyter notebook
Jupyter Notebook Configuration

The script will also offer to configure Jupyter notebook. The main purpose of this is to eliminate the need for the authentication token on remote installations. (Note that this token is import in multi-user environments, but not so much in environments where only a single user logs in.)

  
## setup_conda.sh
#!/usr/bin/env bash

# Michelle L. Gill
# http://michellelynngill.com
# Updated: 2017/05/20
# https://gist.github.com/mlgill/4302c24ad1c8999577fd2f6cd03d8d2b

################# OVERVIEW #################
# This script will setup and install all Conda packages
# in a given path with a given python version
# and environment name. If a conda installation already
# exists at this path, then a new environment with
# the required python version and packages will be created.
# This script works on Mac and Linux.

################# INSTRUCTIONS #################
# 1. Set the path for the conda environment below.
#    This can be a non-existent path for a new
#    installation or an existent path to simply create
#    a new environment.
# 2. Specify the environment name and python version below.
#    This step can be repeated to create multiple environments
#    with different python versions and names, for example.
# 3. Set the list of packages to be installed for the
#    given environment.
# 4. Run this script!


################# USER SET VARIABLES #####################

# Set environment path
# $HOME can be used for a shortcut to your home directory
ENV_PATH="$HOME/miniconda"

# Set environment name
# NOTE: there's not an easy way to determine if an existing environment
# has the same name. Conda itself will throw an error during installation
# of second environment.
# ENV_NAME="scienv2"
ENV_NAME="scienv3"

# Set python version: 2.7, 3.6, etc
# PYTHON_VER=2.7
PYTHON_VER=3.6

# Set packages to be installed (do not list python itself)
# be sure to name/spell them exactly as Conda does or installation will fail
PACKAGES="psutil numpy scipy matplotlib seaborn pandas statsmodels jupyter ipython notebook nbconvert beautifulsoup4 html5lib lxml ipykernel requests flask scikit-learn nltk gensim pymongo"

################# USER SET VARIABLES ABOVE THIS LINE #####################

# BASH prompt colors
BLUE='\033[1;34m'
RED='\033[1;31m'
NC='\033[0m'

# get python major version (2 or 3)
PYTHON_BASE_VER=`echo $PYTHON_VER | cut -d'.' -f1`

echo -e $BLUE"Using Python $PYTHON_VER."$NC

if [[ -e $ENV_PATH ]]; then

    if [[ ! -e $ENV_PATH/bin/conda ]]; then

        # the destination directory does not seem to contain a Conda installation
        # don't want to risk overwriting files, so time to bail...

        echo -e $RED"There doesn't appear to be a Conda installation at $ENV_PATH."$NC
        echo -e $RED"Quitting without installing anything."$NC

        exit 0

    fi

elif [[ ! -e $ENV_PATH ]]; then

    echo -e $BLUE"Conda does not seem to be installed at $ENV_PATH. Installing now."$NC

    # set Linux/Darwin url
    if [[ `uname` == "Darwin" ]]; then
        echo -e $BLUE"Detected Mac OS X."$NC
        OS_VER="MacOSX"
    else
        echo -e $BLUE"Detected Linux."$NC
        OS_VER="Linux"
    fi

    URL="https://repo.continuum.io/miniconda/Miniconda${PYTHON_BASE_VER}-latest-${OS_VER}-x86_64.sh"

    # set install script name
    SCRIPT_NAME="miniconda.sh"

    # download the installation script
    echo -e $BLUE"Downloading Conda installation script."$NC

    if [[ -e $SCRIPT_NAME ]]; then
        rm $SCRIPT_NAME
    fi

    curl -s -o $SCRIPT_NAME $URL

    # create temporary conda installation
    echo -e $BLUE"Creating Conda installation at $ENV_PATH."$NC

    if [[ -e $ENV_PATH ]]; then
        rm -rf $ENV_PATH
    fi

    bash $SCRIPT_NAME -b -f -p $ENV_PATH >> /dev/null

    rm $SCRIPT_NAME

else

    # setup path and create environment
    echo -e $BLUE"Conda seems to be installed already. Skipping installation."$NC

fi

# Setup the environment

unset PYTHONHOME
unset PYTHONPATH

export PATH=$ENV_PATH:$PATH

echo -e $BLUE"Python environment will be based on version $PYTHON_VER"$NC

PACKAGES="python=$PYTHON_VER $PACKAGES"

#$ENV_PATH/bin/conda create --quiet -y -n $ENV_NAME $PACKAGES >> /dev/null
$ENV_PATH/bin/conda create -y -n $ENV_NAME $PACKAGES

$ENV_PATH/bin/conda install -y -n root anaconda-client conda-build

if [[ $?==0 ]]; then
    echo -e $BLUE"Finished creating Conda environment. Any errors directly above regarding psutil or \"command not found\" can probably be ignored."$NC

    # Append environment setup commands to shell startup files if desired and using bash or zsh

    SHELL_NAME=`basename $SHELL`
    SHELLRC="$HOME/.${SHELL_NAME}rc"

    if [[ ($SHELL_NAME == "zsh") || ($SHELL_NAME == "bash") ]]; then

        read -p $'\e[1;34mAppend environment creation variables to shell setup file (.bashrc/.zshrc)? [Y/N]: \e[0m' APPEND

        if [[ ($APPEND == "Y") || ($APPEND == "y") ]]; then

            SHELL_APPEND='\n
            #### Appended by setup_conda.sh #### \n\n
            if [[ -e '"$ENV_PATH"'/bin ]]; then \n
            \t     export PATH='"$ENV_PATH"'/bin:$PATH \n
            \t     source activate '"$ENV_NAME"' \n
            fi \n'

            echo -e $SHELL_APPEND >> $SHELLRC
            echo -e $BLUE"Appended Conda environment setup commands to $SHELLRC"$NC

        else
            echo -e $BLUE"No changes were made to $SHELLRC"$NC
        fi
    fi

    # Jupyter's tokens are a pain when dealing with remote access and not really required
    # if SSH tunnels are used in a trusted environment

    read -p $'\e[1;34mSetup Jupyter config for remote access? [Y/N]: \e[0m' GENERATE_CONFIG

    if [[ ($GENERATE_CONFIG == "Y") || ($GENERATE_CONFIG == "y") ]]; then

        # Backup config file if one exists
        if [[ -e $HOME/.jupyter/jupyter_notebook_config.py ]]; then
            mv -f $HOME/.jupyter/jupyter_notebook_config.py     $HOME/.jupyter/jupyter_notebook_config_bkup.py
        fi

        # Create new config file
        jupyter notebook --generate-config -y

        # Fix settings for config file
        cat $HOME/.jupyter/jupyter_notebook_config.py \
        | sed "s/#c.NotebookApp.password = ''/c.NotebookApp.password = ''/g" \
        | sed "s/#c.NotebookApp.token = '<generated>'/c.NotebookApp.token = ''/g" \
        | sed "s/#c.NotebookApp.open_browser = True/c.NotebookApp.open_browser = False/g" \
        > $HOME/.jupyter/jupyter_notebook_config_new.py

        mv -f $HOME/.jupyter/jupyter_notebook_config_new.py $HOME/.jupyter/jupyter_notebook_config.py

        echo -e $BLUE"Jupyter configuration setup."$NC

    else
        echo -e $BLUE"No Jupyter config file generated."$NC
        echo -e $BLUE"A token from the command line may have to be entered when accessing notebook remotely."$NC
    fi

else
    echo -e $RED"There was an error creating the Conda environment."$NC
fi
	#!/usr/bin/env bash

	# Michelle L. Gill
	# http://michellelynngill.com
	# Updated: 2017/05/20
	# https://gist.github.com/mlgill/4302c24ad1c8999577fd2f6cd03d8d2b

	################# OVERVIEW #################
	# This script will setup and install all Conda packages
	# in a given path with a given python version
	# and environment name. If a conda installation already
	# exists at this path, then a new environment with
	# the required python version and packages will be created.
	# This script works on Mac and Linux.

	################# INSTRUCTIONS #################
	# 1. Set the path for the conda environment below.
	# This can be a non-existent path for a new
	# installation or an existent path to simply create
	# a new environment.
	# 2. Specify the environment name and python version below.
	# This step can be repeated to create multiple environments
	# with different python versions and names, for example.
	# 3. Set the list of packages to be installed for the
	# given environment.
	# 4. Run this script!


	################# USER SET VARIABLES #####################

	# Set environment path
	# $HOME can be used for a shortcut to your home directory
	ENV_PATH="$HOME/miniconda"

	# Set environment name
	# NOTE: there's not an easy way to determine if an existing environment
	# has the same name. Conda itself will throw an error during installation
	# of second environment.
	# ENV_NAME="scienv2"
	ENV_NAME="scienv3"

	# Set python version: 2.7, 3.6, etc
	# PYTHON_VER=2.7
	PYTHON_VER=3.6

	# Set packages to be installed (do not list python itself)
	# be sure to name/spell them exactly as Conda does or installation will fail
	PACKAGES="psutil numpy scipy matplotlib seaborn pandas statsmodels jupyter ipython notebook nbconvert beautifulsoup4 html5lib lxml ipykernel requests flask scikit-learn nltk gensim pymongo"

	################# USER SET VARIABLES ABOVE THIS LINE #####################

	# BASH prompt colors
	BLUE='\033[1;34m'
	RED='\033[1;31m'
	NC='\033[0m'

	# get python major version (2 or 3)
	PYTHON_BASE_VER=`echo $PYTHON_VER \| cut -d'.' -f1`

	echo -e $BLUE"Using Python $PYTHON_VER."$NC

	if [[ -e $ENV_PATH ]]; then

	if [[ ! -e $ENV_PATH/bin/conda ]]; then

	# the destination directory does not seem to contain a Conda installation
	# don't want to risk overwriting files, so time to bail...

	echo -e $RED"There doesn't appear to be a Conda installation at $ENV_PATH."$NC
	echo -e $RED"Quitting without installing anything."$NC

	exit 0

	fi

	elif [[ ! -e $ENV_PATH ]]; then

	echo -e $BLUE"Conda does not seem to be installed at $ENV_PATH. Installing now."$NC

	# set Linux/Darwin url
	if [[ `uname` == "Darwin" ]]; then
	echo -e $BLUE"Detected Mac OS X."$NC
	OS_VER="MacOSX"
	else
	echo -e $BLUE"Detected Linux."$NC
	OS_VER="Linux"
	fi

	URL="https://repo.continuum.io/miniconda/Miniconda${PYTHON_BASE_VER}-latest-${OS_VER}-x86_64.sh"

	# set install script name
	SCRIPT_NAME="miniconda.sh"

	# download the installation script
	echo -e $BLUE"Downloading Conda installation script."$NC

	if [[ -e $SCRIPT_NAME ]]; then
	rm $SCRIPT_NAME
	fi

	curl -s -o $SCRIPT_NAME $URL

	# create temporary conda installation
	echo -e $BLUE"Creating Conda installation at $ENV_PATH."$NC

	if [[ -e $ENV_PATH ]]; then
	rm -rf $ENV_PATH
	fi

	bash $SCRIPT_NAME -b -f -p $ENV_PATH >> /dev/null

	rm $SCRIPT_NAME

	else

	# setup path and create environment
	echo -e $BLUE"Conda seems to be installed already. Skipping installation."$NC

	fi

	# Setup the environment

	unset PYTHONHOME
	unset PYTHONPATH

	export PATH=$ENV_PATH:$PATH

	echo -e $BLUE"Python environment will be based on version $PYTHON_VER"$NC

	PACKAGES="python=$PYTHON_VER $PACKAGES"

	#$ENV_PATH/bin/conda create --quiet -y -n $ENV_NAME $PACKAGES >> /dev/null
	$ENV_PATH/bin/conda create -y -n $ENV_NAME $PACKAGES

	$ENV_PATH/bin/conda install -y -n root anaconda-client conda-build

	if [[ $?==0 ]]; then
	echo -e $BLUE"Finished creating Conda environment. Any errors directly above regarding psutil or \"command not found\" can probably be ignored."$NC

	# Append environment setup commands to shell startup files if desired and using bash or zsh

	SHELL_NAME=`basename $SHELL`
	SHELLRC="$HOME/.${SHELL_NAME}rc"

	if [[ ($SHELL_NAME == "zsh") \|\| ($SHELL_NAME == "bash") ]]; then

	read -p $'\e[1;34mAppend environment creation variables to shell setup file (.bashrc/.zshrc)? [Y/N]: \e[0m' APPEND

	if [[ ($APPEND == "Y") \|\| ($APPEND == "y") ]]; then

	SHELL_APPEND='\n
	#### Appended by setup_conda.sh #### \n\n
	if [[ -e '"$ENV_PATH"'/bin ]]; then \n
	\t export PATH='"$ENV_PATH"'/bin:$PATH \n
	\t source activate '"$ENV_NAME"' \n
	fi \n'

	echo -e $SHELL_APPEND >> $SHELLRC
	echo -e $BLUE"Appended Conda environment setup commands to $SHELLRC"$NC

	else
	echo -e $BLUE"No changes were made to $SHELLRC"$NC
	fi
	fi

	# Jupyter's tokens are a pain when dealing with remote access and not really required
	# if SSH tunnels are used in a trusted environment

	read -p $'\e[1;34mSetup Jupyter config for remote access? [Y/N]: \e[0m' GENERATE_CONFIG

	if [[ ($GENERATE_CONFIG == "Y") \|\| ($GENERATE_CONFIG == "y") ]]; then

	# Backup config file if one exists
	if [[ -e $HOME/.jupyter/jupyter_notebook_config.py ]]; then
	mv -f $HOME/.jupyter/jupyter_notebook_config.py $HOME/.jupyter/jupyter_notebook_config_bkup.py
	fi

	# Create new config file
	jupyter notebook --generate-config -y

	# Fix settings for config file
	cat $HOME/.jupyter/jupyter_notebook_config.py \
	\| sed "s/#c.NotebookApp.password = ''/c.NotebookApp.password = ''/g" \
	\| sed "s/#c.NotebookApp.token = '<generated>'/c.NotebookApp.token = ''/g" \
	\| sed "s/#c.NotebookApp.open_browser = True/c.NotebookApp.open_browser = False/g" \
	> $HOME/.jupyter/jupyter_notebook_config_new.py

	mv -f $HOME/.jupyter/jupyter_notebook_config_new.py $HOME/.jupyter/jupyter_notebook_config.py

	echo -e $BLUE"Jupyter configuration setup."$NC

	else
	echo -e $BLUE"No Jupyter config file generated."$NC
	echo -e $BLUE"A token from the command line may have to be entered when accessing notebook remotely."$NC
	fi

	else
	echo -e $RED"There was an error creating the Conda environment."$NC
	fi