Skip to content

Instantly share code, notes, and snippets.

@justincbagley
Last active May 3, 2017 17:18
Show Gist options
  • Save justincbagley/ecab6c39b62f89458b8ed3fab98fbeb2 to your computer and use it in GitHub Desktop.
Save justincbagley/ecab6c39b62f89458b8ed3fab98fbeb2 to your computer and use it in GitHub Desktop.
How to Install and Run Modified ```∂a∂i``` v1.6.3 from Tine *et al.* (2014)

How To Install and Run Modified ∂a∂i v1.6.3 from Tine et al. (2014)

This file describes how I installed Tine et al.'s modified version of ∂a∂i software on the BYU FSL supercomputing cluster marylou, and got it working, on April 15, 2017. Tine et al.'s modification of ∂a∂i permits models of heterogeneous migration between lineages, consistent with introgression of specific regions of the genome.

STEP #1. Download ∂a∂i mod distribution.

Tine et al.'s modification of ∂a∂i v1.6.3 is distributed in a "dadi-1.6.3_modif" folder containing the distro, which is available as part of a blog post at this SEA Lab website. Go to the bottom of the blog post and then download the modified ∂a∂i archive from this link.

STEP #2. Python virtual environment setup on supercomputer.

2a. Uninstall the modules if they're present, then reinstall virtualenv with conda and install virtualenvwrapper with pip:

$ pip uninstall virtualenv; # (y)
$ conda uninstall virtualenv; # (y)
$ pip uninstall virtualenvwrapper; # (y)
$ #
$ conda install virtualenv; # (y)
$ pip install virtualenvwrapper; # (y)
$ source ~/miniconda2/bin/virtualenvwrapper.sh

2b. Add the following lines to your ~/.bashrc or ~/.bash_profile script, source the script you just edited, then exit/close session and open a new session:

##--Added after installing Python package virtualenvwrapper (and virtualenv dependency), per instructions at
##--URL: http://docs.python-guide.org/en/latest/dev/virtualenvs/
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source ~/miniconda2/bin/virtualenvwrapper.sh

Some other code from the developers/online, which I do not use and don't recommend, unless it exactly fits your needs, desires, and system. The following doesn't work for me because I don't want to use an "Envs" folder for my virtual environments, and I don't have admin privileges, so I don't have access to /usr/local or /usr/local/bin on BYU supercomputer.

WORKON_HOME="${WORKON_HOME}:~/Envs"
export WORKON_HOME
source /usr/local/bin/virtualenvwrapper.sh

NOTE: You may need to manually edit the final source line. So, before or after editing your bash file(s) as per above, first run a check to locate the corresponding shell script, e.g. by doing $ find . -name "*virtualenvwrapper.sh" -type f from within your working dir, or $HOME dir (preferred).

Check the virtual environment setup by attempting a virtualenv named "temp", as follows:

$ cd ~
$ mkvirtualenv temp
New python executable in /zhome/bagle004/.virtualenvs/temp/bin/python
copying /fslhome/bagle004/miniconda2/bin/python => /zhome/bagle004/.virtualenvs/temp/bin/python
copying /fslhome/bagle004/miniconda2/bin/../lib/libpython2.7.so.1.0 => /zhome/bagle004/.virtualenvs/temp/lib/libpython2.7.so.1.0
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/temp/bin/predeactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/temp/bin/postdeactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/temp/bin/preactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/temp/bin/postactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/temp/bin/get_env_details
Error: deactivate must be sourced. Run 'source deactivate'
instead of 'deactivate'.

(temp) -bash-4.1$ source deactivate
(temp) -bash-4.1$ deactivate

STEP #3. Make dadi-mod virtual environment, and then check it.

Make the environment and populate it with relevant stuff.

-bash-4.1$ mkvirtualenv dadi-mod
New python executable in /zhome/bagle004/.virtualenvs/dadi-mod/bin/python
copying /fslhome/bagle004/miniconda2/bin/python => /zhome/bagle004/.virtualenvs/dadi-mod/bin/python
copying /fslhome/bagle004/miniconda2/bin/../lib/libpython2.7.so.1.0 => /zhome/bagle004/.virtualenvs/dadi-mod/lib/libpython2.7.so.1.0
Installing setuptools, pip, wheel...done.
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/dadi-mod/bin/predeactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/dadi-mod/bin/postdeactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/dadi-mod/bin/preactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/dadi-mod/bin/postactivate
virtualenvwrapper.user_scripts creating /fslhome/bagle004/.virtualenvs/dadi-mod/bin/get_env_details
Error: deactivate must be sourced. Run 'source deactivate'
instead of 'deactivate'.

(dadi-mod) -bash-4.1$ 

STEP #4. Install dadi-mod dependencies into dadi-mod virtual environment Python stash.

##--Check Python in the dadi-mod env, and you'll see that there's almost nothing modules-wise:
(dadi-mod) -bash-4.1$ cd ~/.virtualenvs/dadi-mod/lib/python2.7/
(dadi-mod) -bash-4.1$ ls
#
##--Install SciPy and NumPy for the dadi-mod env:
(dadi-mod) -bash-4.1$ conda install scipy --prefix=~/.virtualenvs/dadi-mod/
(dadi-mod) -bash-4.1$ conda install numpy --prefix=~/.virtualenvs/dadi-mod/

Tests:

##--Check Python in the dadi-mod env again, and you'll see that now there's LOTS of goodies in there!:
(dadi-mod) -bash-4.1$ cd ~/.virtualenvs/dadi-mod/lib/python2.7/
(dadi-mod) -bash-4.1$ ls

STEP #5. Install dadi-1.6.3-mod software from modified archive downloaded under STEP #1 above, and check it.

Do install with prefix:

(dadi-mod) -bash-4.1$ cd ~/compute/dadi-1.6.3_modif/; python setup.py install --user

Run tests script within the dadi-mod archive:

(dadi-mod) -bash-4.1$ cd ~/compute/dadi-1.6.3_modif/tests
(dadi-mod) -bash-4.1$ ls
admixture_test_manual.py  run_tests.py        test_Admixture.pyc  test_Optimization.py   test_phi1D.pyc    test_Spectrum.py     test_Tridiagonal.pyc
build                     test_2D_frozen1.fs  test_Freezing.py    test_Optimization.pyc  test_Results.py   test_Spectrum.pyc
IM.fs                     test_Admixture.py   test_Freezing.pyc   test_phi1D.py          test_Results.pyc  test_Tridiagonal.py
(dadi-mod) -bash-4.1$ python run_tests.py 

Open Python and make sure everything imports correctly:

(dadi-mod) -bash-4.1$ python
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:09:15) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import numpy
>>> import scipy
>>> import dadi
>>> exit()

STEP #6. Now you're ready to run heterogeneous migration models in ∂a∂i!

References

  • Tine M, Kuhl H, Gagnaire P-A et al. (2014) European sea bass genome and its variation provide insights into adaptation to euryhalinity and speciation. Nature Communications, 5.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment