Install Ensembl's VEP v84 with various caches for running offline

ensembl_vep_84_with_offline_cache.md

Ensembl's VEP (Variant Effect Predictor) is popular for how it picks a single effect per gene as detailed here, its CLIA-compliant HGVS variant format, and Sequence Ontology nomenclature for variant effects.

To follow these instructions, we'll assume you have these packaged essentials installed:

sudo yum install -y curl rsync tar make perl perl-core
## OR ##
sudo apt-get install -y curl rsync tar make perl perl-base

VEP requires Perl 5.10 or newer, but I'd strongly recommend at least 5.18. If your system Perl is outdated, then follow this gist to set up Perl 5.22. Once that's done, set temporary shell variables pointing to where Perl and its libraries live. Change this as needed for your system:

export PERL_PATH=/opt/common/CentOS_6-dev/perl/perl-5.22.0
export PERL_BIN=/opt/common/CentOS_6-dev/perl/perl-5.22.0/bin/perl

Handle VEP's Perl dependencies using cpanminus to install them under $PERL_PATH:

curl -L http://cpanmin.us | $PERL_BIN - --notest -l $PERL_PATH LWP LWP::Simple LWP::Protocol::https Archive::Extract Archive::Tar Archive::Zip CGI DBI Time::HiRes DBD::mysql Encode File::Copy::Recursive Perl::OSType Module::Metadata version

Set PERL5LIB to find those libraries. Add this to the end of your ~/.bashrc to make it persistent:

export PERL5LIB=$PERL_PATH/lib/perl5:$PERL_PATH/lib/perl5/x86_64-linux

Create temporary shell variables pointing to where we'll store VEP and its cache data (non default paths can be used, but specify --vep-path and --vep-data when running vcf2maf or maf2maf):

export VEP_PATH=/opt/common/CentOS_6-dev/vep/v84
export VEP_DATA=/opt/common/CentOS_6-dev/vep/v84

Download the v84 release of VEP:

mkdir $VEP_PATH $VEP_DATA; cd $VEP_PATH
curl -LO https://github.com/Ensembl/ensembl-tools/archive/release/84.tar.gz
tar -zxf 84.tar.gz --starting-file variant_effect_predictor --transform='s|.*/|./|g'

Add that path to PERL5LIB, and the htslib subfolder to PATH where tabix will be installed:

export PERL5LIB=$VEP_PATH:$PERL5LIB
export PATH=$VEP_PATH/htslib:$PATH

Download and unpack VEP's offline cache for GRCh37, GRCh38, and GRCm38:

rsync -zvh rsync://ftp.ensembl.org/ensembl/pub/release-84/variation/VEP/homo_sapiens_vep_84_GRCh{37,38}.tar.gz $VEP_DATA
rsync -zvh rsync://ftp.ensembl.org/ensembl/pub/release-84/variation/VEP/mus_musculus_vep_84_GRCm38.tar.gz $VEP_DATA
cat $VEP_DATA/*_vep_84_GRC{h37,h38,m38}.tar.gz | tar -izxf - -C $VEP_DATA

Install the Ensembl API, the reference FASTAs for GRCh37/GRCh38/GRCm38, and some neat VEP plugins:

$PERL_BIN INSTALL.pl --AUTO afp --SPECIES homo_sapiens --ASSEMBLY GRCh37 --PLUGINS ExAC,UpDownDistance --DESTDIR $VEP_PATH --CACHEDIR $VEP_DATA
$PERL_BIN INSTALL.pl --AUTO afp --SPECIES homo_sapiens --ASSEMBLY GRCh38 --PLUGINS ExAC,UpDownDistance --DESTDIR $VEP_PATH --CACHEDIR $VEP_DATA
$PERL_BIN INSTALL.pl --AUTO afp --SPECIES mus_musculus --ASSEMBLY GRCm38 --PLUGINS UpDownDistance --DESTDIR $VEP_PATH --CACHEDIR $VEP_DATA

Convert the offline cache for use with tabix, that significantly speeds up the lookup of known variants:

$PERL_BIN convert_cache.pl --species homo_sapiens --version 84_GRCh37,84_GRCh38 --dir $VEP_DATA
$PERL_BIN convert_cache.pl --species mus_musculus --version 84_GRCm38 --dir $VEP_DATA

Download and index a custom ExAC r0.3 VCF, that skips variants overlapping known somatic hotspots:

curl -L https://googledrive.com/host/0B6o74flPT8FAYnBJTk9aTF9WVnM > $VEP_DATA/ExAC.r0.3.sites.minus_somatic.vcf.gz
tabix -p vcf $VEP_DATA/ExAC.r0.3.sites.minus_somatic.vcf.gz

Test running VEP in offline mode with the ExAC plugin, on the provided sample GRCh37 VCF:

$PERL_BIN variant_effect_predictor.pl --species homo_sapiens --assembly GRCh37 --offline --no_progress --everything --shift_hgvs 1 --check_existing --check_alleles --total_length --allele_number --no_escape --xref_refseq --dir $VEP_DATA --fasta $VEP_DATA/homo_sapiens/84_GRCh37/Homo_sapiens.GRCh37.75.dna.primary_assembly.fa.gz --plugin ExAC,$VEP_DATA/ExAC.r0.3.sites.minus_somatic.vcf.gz --input_file example_GRCh37.vcf --output_file example_GRCh37.vep.txt

I've run into a couple of issues

1. $PERL_BIN convert_cache.pl --species mus_musculus --version 84_GRCm38 --dir $VEP_DATA
   2018-07-09 09:27:32 - Processing mus_musculus 2018-07-09 09:27:32 - Processing version 84_GRCm38 Can't use an undefined value as an ARRAY reference at convert_cache.pl line 188.
2. tabix -p vcf $VEP_DATA/ExAC.r0.3.sites.minus_somatic.vcf.gz
   Not a BGZF file: /opt/vcf2maf/vep/ExAC.r0.3.sites.minus_somatic.vcf.gz tbx_index_build failed: /opt/vcf2maf/vep/ExAC.r0.3.sites.minus_somatic.vcf.gz
   Any help would be appreciated. Thanks.