The installation guide on https://github.com/facebookresearch/DensePose seemed pretty lacking and I spent a bunch of time trying to figure out how to install so I kept track of my steps. Voila.
Go to your EC2 Dashboard
Launch a Ubuntu 16.04 instance
Choose an instance type that has a GPU. I think g2.2xlarge is the cheapest one.
Before you launch it, make sure you change the storage settings so that the root volume is 100GB not 8GB.
Launch it with an existing key pair if you have one, or create a new one if you don’t
SSH into you instance after it launches:
ssh -i ~/.ssh/YOUR_KEY_PAIR.pem ubuntu@YOUR-EC2-ADDRESS.compute.amazonaws.com
Once you are SSHed in, install a bunch of dependencies.
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
build-essential \
cmake \
git \
libgoogle-glog-dev \
libgtest-dev \
libiomp-dev \
libleveldb-dev \
liblmdb-dev \
libopencv-dev \
libopenmpi-dev \
libsnappy-dev \
libprotobuf-dev \
openmpi-bin \
openmpi-doc \
protobuf-compiler \
python-dev \
python-pip
sudo apt-get install python-setuptools
sudo pip install future numpy protobuf enum networkx cython
sudo apt-get install -y --no-install-recommends libgflags-dev
Now install CUDA
sudo apt-get update && sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda
Download cuDNN v6.0 for CUDA 8.0 here. You will have to make an NVIDIA developer account. Find the link that says "cuDNN v6.0 Library for Linux" and download it to your Downloads folder.
Open a new Terminal tab and copy it over to your EC2 instance.
scp -i ~/.ssh/YOUR_KEY_PAIR.pem ~/Downloads/cudnn-8.0-linux-x64-v6.0.tgz ubuntu@YOUR-EC2-ADDRESS.compute.amazonaws.com:~/
Go back to your EC2 instance tab and install.
sudo tar -xzf cudnn-8.0-linux-x64-v6.0.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v6.0.tgz && sudo ldconfig
(skip this section if you remembered to edit your storage settings when launching the instance)
densepose is huge and the disk that comes with the g2.2xlarge is too small so you have to expand the disk size.
Open your EC2 Dashboard. Click on the instance you just launched. Find Block Devices, and click on "/dev/sda1" and then click on the EBS ID link. The click Actions, and select Modify Volume. I added 100GB -- not exactly sure how much you need. This takes a few minutes to take effect. Type lsblk in your Terminal to see if the disk has been expanded.
Next you need to expand your partition. Run sudo growpart /dev/xvda 1
I had to reboot the instance for this to take effect.
Run the following to install Caffe2 first
export PYTHONPATH=/usr/local
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/pytorch/build
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
git clone --recursive https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init
# Create a directory to put Caffe2's build files in
mkdir build && cd build
# Configure Caffe2's build
# This looks for packages on your machine and figures out which functionality
# to include in the Caffe2 installation. The output of this command is very
# useful in debugging.
cmake ..
sudo make install -j8
Run python pytorch/caffe2/python/operator_test/reshape_ops_test.py and make sure this works.
Install COCO dataset
cd
sudo git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI/
sudo make install
Install detectron
cd
git clone https://github.com/facebookresearch/detectron
sudo pip install -r detectron/requirements.txt
cd detectron
make
Test to make sure detectron installed properly
cd
python detectron/detectron/tests/test_spatial_narrow_as_op.py
Install densepose
git clone https://github.com/facebookresearch/densepose
sudo pip install -r densepose/requirements.txt
cd densepose && make
make ops
Test to make sure densepose works
python2 $DENSEPOSE/detectron/tests/test_zero_even_op.py
Enjoy :)
Hi, I installed caffe2 and densepose based on your topic but I can't run test_zero_even_op.py because of undefined symbol
$ python2 detectron/tests/test_zero_even_op.py
E0919 11:45:22.971385 18390 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971402 18390 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971419 18390 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/home/aiteam/workspace/DensePose/detectron/utils/c2.py", line 40, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/usr/lib/python2.7/ctypes/init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/aiteam/workspace/DensePose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE
My system is:
Ubuntu 16.04
CUDA 8.0
cuDNN 6.0
Python 2.7.12