Skip to content

Instantly share code, notes, and snippets.

@canzhiye canzhiye/install.md
Last active Dec 14, 2018

Embed
What would you like to do?
how to install densepose

installing densepose

The installation guide on https://github.com/facebookresearch/DensePose seemed pretty lacking and I spent a bunch of time trying to figure out how to install so I kept track of my steps. Voila.


Go to your EC2 Dashboard

Launch a Ubuntu 16.04 instance

Choose an instance type that has a GPU. I think g2.2xlarge is the cheapest one.

Before you launch it, make sure you change the storage settings so that the root volume is 100GB not 8GB.

Launch it with an existing key pair if you have one, or create a new one if you don’t

SSH into you instance after it launches:

ssh -i ~/.ssh/YOUR_KEY_PAIR.pem ubuntu@YOUR-EC2-ADDRESS.compute.amazonaws.com

Once you are SSHed in, install a bunch of dependencies.

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      libprotobuf-dev \
      openmpi-bin \
      openmpi-doc \
      protobuf-compiler \
      python-dev \
      python-pip  
      
sudo apt-get install python-setuptools
sudo pip install future numpy protobuf enum networkx cython

sudo apt-get install -y --no-install-recommends libgflags-dev

Now install CUDA

sudo apt-get update && sudo apt-get install wget -y --no-install-recommends
wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_8.0.61-1_amd64.deb"
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda

Download cuDNN v6.0 for CUDA 8.0 here. You will have to make an NVIDIA developer account. Find the link that says "cuDNN v6.0 Library for Linux" and download it to your Downloads folder.

Open a new Terminal tab and copy it over to your EC2 instance.

scp -i ~/.ssh/YOUR_KEY_PAIR.pem ~/Downloads/cudnn-8.0-linux-x64-v6.0.tgz ubuntu@YOUR-EC2-ADDRESS.compute.amazonaws.com:~/

Go back to your EC2 instance tab and install.

sudo tar -xzf cudnn-8.0-linux-x64-v6.0.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v6.0.tgz && sudo ldconfig

(skip this section if you remembered to edit your storage settings when launching the instance)

densepose is huge and the disk that comes with the g2.2xlarge is too small so you have to expand the disk size.

Open your EC2 Dashboard. Click on the instance you just launched. Find Block Devices, and click on "/dev/sda1" and then click on the EBS ID link. The click Actions, and select Modify Volume. I added 100GB -- not exactly sure how much you need. This takes a few minutes to take effect. Type lsblk in your Terminal to see if the disk has been expanded.

Next you need to expand your partition. Run sudo growpart /dev/xvda 1

I had to reboot the instance for this to take effect.


Run the following to install Caffe2 first

export PYTHONPATH=/usr/local
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/pytorch/build
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

git clone --recursive https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init

# Create a directory to put Caffe2's build files in
mkdir build && cd build

# Configure Caffe2's build
# This looks for packages on your machine and figures out which functionality
# to include in the Caffe2 installation. The output of this command is very
# useful in debugging.
cmake ..

sudo make install -j8

Run python pytorch/caffe2/python/operator_test/reshape_ops_test.py and make sure this works.


Install COCO dataset

cd
sudo git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI/
sudo make install

Install detectron

cd
git clone https://github.com/facebookresearch/detectron
sudo pip install -r detectron/requirements.txt 
cd detectron
make

Test to make sure detectron installed properly

cd
python detectron/detectron/tests/test_spatial_narrow_as_op.py 

Install densepose

git clone https://github.com/facebookresearch/densepose
sudo pip install -r densepose/requirements.txt
cd densepose && make
make ops

Test to make sure densepose works

python2 $DENSEPOSE/detectron/tests/test_zero_even_op.py

Enjoy :)

Fun thing I made with this

@bH-AGC

This comment has been minimized.

Copy link

commented Sep 17, 2018

Dear canzhiye,

I try to install Densepose on an EC2 Ubuntu Instance but unfortunately I have an issue at this step 👍

screen shot 2018-09-17 at 12 42 34

Could you help me please?

Kind regards,
Hadrien

@bH-AGC

This comment has been minimized.

Copy link

commented Sep 17, 2018

I found the solution :

  • sudo -H pip install pyyalm
  • sudo -H pip install typing
@hoangtnm

This comment has been minimized.

Copy link

commented Sep 19, 2018

Hi, I installed caffe2 and densepose based on your topic but I can't run test_zero_even_op.py because of undefined symbol

$ python2 detectron/tests/test_zero_even_op.py
E0919 11:45:22.971385 18390 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971402 18390 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971419 18390 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/home/aiteam/workspace/DensePose/detectron/utils/c2.py", line 40, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/usr/lib/python2.7/ctypes/init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/aiteam/workspace/DensePose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

My system is:
Ubuntu 16.04
CUDA 8.0
cuDNN 6.0
Python 2.7.12

@bH-AGC

This comment has been minimized.

Copy link

commented Sep 19, 2018

Hi, I installed caffe2 and densepose based on your topic but I can't run test_zero_even_op.py because of undefined symbol

$ python2 detectron/tests/test_zero_even_op.py
E0919 11:45:22.971385 18390 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971402 18390 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0919 11:45:22.971419 18390 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/home/aiteam/workspace/DensePose/detectron/utils/c2.py", line 40, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/home/aiteam/workspace/pytorch/build/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/usr/lib/python2.7/ctypes/init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/aiteam/workspace/DensePose/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

My system is:
Ubuntu 16.04
CUDA 8.0
cuDNN 6.0
Python 2.7.12

Hi @hoangtnm,

I have exactly the same error of you and I'm looking for solutions on different forum but nothing...

E0919 08:23:22.011670 2390 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
File "detectron/tests/test_zero_even_op.py", line 117, in
c2_utils.import_custom_ops()
File "/home/ubuntu/detectron/detectron/utils/c2.py", line 50, in import_custom_ops
dyndep.InitOpsLibrary(custom_ops_lib)
File "/usr/local/lib/python2.7/dist-packages/caffe2/python/dyndep.py", line 35, in InitOpsLibrary
_init_impl(name)
File "/usr/local/lib/python2.7/dist-packages/caffe2/python/dyndep.py", line 48, in _init_impl
ctypes.CDLL(path)
File "/usr/lib/python2.7/ctypes/init.py", line 362, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/ubuntu/detectron/build/libcaffe2_detectron_custom_ops_gpu.so: undefined symbol: _ZN6google8protobuf8internal9ArenaImpl28AllocateAlignedAndAddCleanupEmPFvPvE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.