Skip to content

Instantly share code, notes, and snippets.

@skeeet

skeeet/setup.md

Forked from fortunto2/setup.md
Created March 13, 2017 20:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save skeeet/14f1d91f410d460a34547d203b53ddba to your computer and use it in GitHub Desktop.
Save skeeet/14f1d91f410d460a34547d203b53ddba to your computer and use it in GitHub Desktop.
Download ZIP
Setup Amazon AWS EC2 g2.2xlarge instance with OpenCV 3.1, Cuda 7.5, ffmpeg, OpenFace
Raw
setup.md

Ubuntu 14 on AWS GPU instance

Verify You Have a CUDA-Capable GPU

lspci | grep -i nvidia

Souce

Start

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install -y build-essential git libcurl4-openssl-dev libatlas-base-dev libopencv-dev python-numpy unzip

sudo apt-get install -y opencl-headers build-essential protobuf-compiler \
    libprotoc-dev libboost-all-dev libleveldb-dev hdf5-tools libhdf5-serial-dev \
    libopencv-core-dev  libopencv-highgui-dev libsnappy-dev libsnappy1 \
    libatlas-base-dev cmake libstdc++6-4.8-dbg libgoogle-glog0 libgoogle-glog-dev \
    libgflags-dev liblmdb-dev git python-pip gfortran

Update Linux kernel on AWS

sudo apt-get install linux-image-extra-virtual

Disable nouveau

nouveau conflicts with NVIDIA’s kernel module on AWS.

sudo vi /etc/modprobe.d/blacklist-nouveau.conf

    blacklist nouveau
    blacklist lbm-nouveau
    options nouveau modeset=0
    alias nouveau off
    alias lbm-nouveau off

echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
sudo update-initramfs -u
sudo reboot

sudo apt-get install -y linux-image-extra-`uname -r` linux-headers-`uname -r` linux-image-`uname -r`

Install CUDA repository

wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
sudo apt-get update

sudo apt-get install -y cuda
sudo apt-get clean
sudo reboot

Important: Please reboot the instance for loading the driver

Verify CUDA

nvidia-smi

cuDNN

One can apply for the developer program here https://developer.nvidia.com/cudnn. When approved, download cuDNN for Linux (either v5), upload the cuDNN package from the local computer to the instance, and install cuDNN:

https://developer.nvidia.com/rdp/assets/cudnn-7.5-linux-x64-v5.0-ga-tgz

tar -zxf cudnn-7.0-linux-x64-v4.0-rc.tgz #or cudnn-7.0-linux-x64-v3.0-prod.tgz
cd cuda
sudo cp lib64/* /usr/local/cuda/lib64/
sudo cp include/cudnn.h /usr/local/cuda/include/

PATH

~/.profile

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

FFMPEG

wget 'http://developer.download.nvidia.com/compute/redist/ffmpeg/1511-patch/cudautils.zip'
wget 'http://ffmpeg.org/releases/ffmpeg-3.1.tar.bz2'
wget 'http://kuzko.com/dl/ffmpeg_NVIDIA_gpu_acceleration.3.0.2.patch'

x264

git clone git://git.videolan.org/x264.git
cd x264

./configure --disable-cli --enable-static --enable-shared --enable-strip
make -j
make install
ldconfig

Nvenc

wget 'https://developer.nvidia.com/video-sdk-601' -O 'video-sdk-601.zip'
unzip video-sdk-601.zip;
sudo /bin/cp -prf nvidia_video_sdk_6.0.1/Samples/common/inc/*.h /usr/local/include

cudautils

wget http://developer.download.nvidia.com/compute/redist/ffmpeg/1511-patch/cudautils.zip

unzip cudautils.zip
cd cudautils
make

FFMPEG

git clone git://source.ffmpeg.org/ffmpeg.git

wget http://developer.download.nvidia.com/compute/redist/ffmpeg/1511-patch/ffmpeg_NVIDIA_gpu_acceleration.patch
cd ffmpeg
git apply ../ffmpeg_NVIDIA_gpu_acceleration.patch
cd ..
mkdir ffmpeg_build
cd ffmpeg_build


../ffmpeg/configure --enable-nonfree --enable-nvenc --enable-nvresize --extra-cflags=-I../cudautils --extra-ldflags=-L../cudautils --enable-gpl --enable-libx264

#--enable-nvresize пришлось удалить

make -j

./ffmpeg -encoders | grep 264

frei0r

sudo apt-get install frei0r-plugins

add Swap

https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-14-04

sudo swapon -s

#8gb
sudo dd if=/dev/zero of=/swapfile bs=1G count=8
sudo chmod 600 /swapfile
ls -lh /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

sudo swapon -s

sudo nano /etc/fstab
# add - "/swapfile   none    swap    sw    0   0"

GCC 4.9

http://askubuntu.com/questions/466651/how-do-i-use-the-latest-gcc-on-ubuntu

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install g++-4.9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 60 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9

GStreamer

sudo add-apt-repository ppa:gstreamer-developers/ppa
sudo apt-get update
sudo apt-get install gstreamer1.0*

OPENCV

http://blog.aicry.com/ubuntu-14-04-install-opencv-with-cuda/

sudo apt-get install libopenblas-dev liblapack-dev
sudo apt-get install git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev checkinstall

sudo apt-get install libopencv-dev build-essential checkinstall cmake pkg-config yasm libtiff4-dev libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils

sudo apt-get -y install libopencv-dev build-essential cmake git libgtk2.0-dev pkg-config python-dev python-numpy libdc1394-22 libdc1394-22-dev libjpeg-dev libpng12-dev libtiff4-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev libtbb-dev libqt4-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils unzip

wget https://github.com/Itseez/opencv/archive/3.1.0.zip

cd opencv-3.1.0
sudo mkdir build
cd build

sudo apt-get install qt-sdk

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_SHARED_LIBS=OFF ..

#or
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 ..  

ccmake ..
# WITH_NVCUVID - ON
# WITH_CUBLAS - ON

sudo make -j
sudo make install

problem solve

http://askubuntu.com/questions/691889/opencv-with-cuda-on-ubuntu-14-04-installation-error

sudo ln -s /usr/lib/nvidia-352/libnvcuvid.so /usr/lib/libnvcuvid.so
sudo ln -s /usr/lib/nvidia-352/libnvcuvid.so.1 /usr/lib/libnvcuvid.so.1

if undefined reference to avformat_get_mov_video_tags

sudo apt-get remove libavutil-dev libavutil-extra-51

OpenFace

https://github.com/TadasBaltrusaitis/OpenFace/wiki/Unix-Installation

Boost

sudo apt-get install libboost1.55-all-dev
#or
sudo apt-get install libboost-all-dev

OpenFace

git clone https://github.com/TadasBaltrusaitis/OpenFace.git

cd OpenFace
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE ..

make
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment