-
-
Save partnercloudsupport/9823e1a78822f172552d4c7e9e533d3c to your computer and use it in GitHub Desktop.
Install script of caffe2 and detectron on AWS EC2 instance with Deep Learning Base AMI
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Install script of Caffe2 and Detectron on AWS EC2 | |
# | |
# Tested environment: | |
# - AMI: Deep Learning Base AMI (Ubuntu) Version 3.0 - ami-38c87440 (CUDA is already installed) | |
# - Instance: p3.2xlarge (V100 * 1) | |
# - Caffe2: https://github.com/caffe2/caffe2/commit/e1f614a5f8ae92f4ecb828e1d5f84d2cd1fe12bd | |
# - Detectron: https://github.com/facebookresearch/Detectron/commit/a22302de27f9004422a96414ed4088d05c664978 | |
# | |
# Usage: | |
# Launch a fresh EC2 instance, and run the following command. | |
# $ bash install_caffe2_detectroh.sh | |
# | |
# Test: | |
# $ cd ~/work/detectron | |
# $ python2 tests/test_spatial_narrow_as_op.py | |
# | |
# Run samples: | |
# Run the following commands and see the results in /tmp/detectron-visualizations | |
# $ cd ~/work/detectron | |
# $ python2 tools/infer_simple.py \ | |
# --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \ | |
# --output-dir /tmp/detectron-visualizations \ | |
# --image-ext jpg \ | |
# --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \ | |
# demo | |
# | |
# Note that: | |
# - In the Deep Learning AMI (Version 4.0), CUDA and caffe2 are already installed. But the caffe2 in the AMI is | |
# a bit old version and does not include some modules required for Detectron. | |
# So this script supposes that an AMI is Deep Learning "Base" AMI (Version 3.0), where only CUDA is installed. | |
# - Manual configuration of setup.py is not a recommended way. Any suggestions are welcome. | |
INSTALL_DIR=~/work | |
### Install Caffe2 | |
sudo apt-get update | |
sudo apt-get install -y --no-install-recommends \ | |
build-essential \ | |
cmake \ | |
git \ | |
libgoogle-glog-dev \ | |
libgtest-dev \ | |
libiomp-dev \ | |
libleveldb-dev \ | |
liblmdb-dev \ | |
libopencv-dev \ | |
libopenmpi-dev \ | |
libsnappy-dev \ | |
libprotobuf-dev \ | |
openmpi-bin \ | |
openmpi-doc \ | |
protobuf-compiler \ | |
python-dev \ | |
python-pip | |
sudo pip install \ | |
future \ | |
numpy \ | |
protobuf | |
sudo apt-get install -y --no-install-recommends libgflags-dev | |
mkdir -p $INSTALL_DIR && cd $INSTALL_DIR | |
git clone --recursive https://github.com/caffe2/caffe2.git && cd caffe2 | |
mkdir -p build && cd build | |
cmake .. | |
sudo make install -j6 | |
# Export paths | |
echo "export PYTHONPATH=/usr/local:\$PYTHONPATH" >> ~/.bashrc | |
echo "export PYTHONPATH=\$PYTHONPATH:${INSTALL_DIR}/caffe2/build" >> ~/.bashrc | |
echo "export LD_LIBRARY_PATH=/usr/local/lib:\$LD_LIBRARY_PATH" >> ~/.bashrc | |
source activate ~/.bashrc | |
### Install Detectron | |
sudo pip install numpy pyyaml matplotlib opencv-python setuptools Cython mock scipy | |
# First, install coco api | |
COCOAPI=$INSTALL_DIR/cocoapi | |
git clone https://github.com/cocodataset/cocoapi.git $COCOAPI | |
cd $COCOAPI/PythonAPI | |
# https://github.com/facebookresearch/Detectron/issues/105 | |
# https://github.com/cocodataset/cocoapi/issues/94 | |
# Based on the comments above, insert `extra_link_args=['-L/usr/lib/x86_64-linux-gnu/']` in setup.py | |
sed -i -e "/extra_compile_args/a \ extra_link_args=['-L/usr/lib/x86_64-linux-gnu/']," setup.py | |
# Install into global site-packages | |
sudo make install | |
# Next, install detectron | |
DETECTRON=$INSTALL_DIR/detectron | |
git clone https://github.com/facebookresearch/detectron $DETECTRON | |
cd $DETECTRON/lib | |
# Add `extra_link_args ... ` in setup.py in the same manner as COCOAPI | |
sed -i -e "/name='utils.cython/a \ extra_link_args=['-L/usr/lib/x86_64-linux-gnu/']," setup.py | |
make |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment