Skip to content

Instantly share code, notes, and snippets.

@pjspillai
Last active September 11, 2022 21:25
  • Star 3 You must be signed in to star a gist
  • Fork 3 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save pjspillai/87625e1b6a129335eb363a4131a301f2 to your computer and use it in GitHub Desktop.
OS: Ubuntu 18.04
Others: Opencv, NCCL, CUDA 9.2, CUDNN
NOTE: For Ubuntu 17.04/ 18.04, there's an alternate way to directly install Caffe via apt-get install caffe-cpu and caffe-cuda
I am installing from source so that I can with other caffe algorithms that demand the existence of a CAFFE_ROOT directory
Modify Makefile.config file
--------------------
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_61,code=compute_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
# $(ANACONDA_HOME)/include/python2.7 \
# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
----------------------------------------------------------------
make all -j12
make test -j12
sudo ldconfig /usr/local/cuda/lib64/
make runtest -j12
It ran smoothly without a glitch!
@yushenglive
Copy link

(1) In my case, I found on 18.04 "sudo apt-get install libboost-all-dev" doesn't work:
syu@VAN-2000-NEW:~$ sudo apt-get install libboost-all-dev
[sudo] password for syu:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help resolve the situation:

The following packages have unmet dependencies:
libboost-all-dev : Depends: libboost-iostreams-dev but it is not going to be installed
Depends: libboost-regex-dev but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

(2) I have to install libboost-dev, then it allerts:
syu@VAN-2000-NEW:~/ml/caffe$ sudo apt-get install libboost-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libboost1.65-dev
Suggested packages:
libboost-doc libboost1.65-doc libboost-atomic1.65-dev libboost-chrono1.65-dev
libboost-container1.65-dev libboost-context1.65-dev libboost-coroutine1.65-dev
libboost-date-time1.65-dev libboost-exception1.65-dev libboost-fiber1.65-dev
libboost-filesystem1.65-dev libboost-graph1.65-dev libboost-graph-parallel1.65-dev
libboost-iostreams1.65-dev libboost-locale1.65-dev libboost-log1.65-dev
libboost-math1.65-dev libboost-mpi1.65-dev libboost-mpi-python1.65-dev
libboost-numpy1.65-dev libboost-program-options1.65-dev libboost-python1.65-dev
libboost-random1.65-dev libboost-regex1.65-dev libboost-serialization1.65-dev
libboost-signals1.65-dev libboost-stacktrace1.65-dev libboost-system1.65-dev
libboost-test1.65-dev libboost-thread1.65-dev libboost-timer1.65-dev
libboost-type-erasure1.65-dev libboost-wave1.65-dev libboost1.65-tools-dev libmpfrc++-dev
libntl-dev
The following NEW packages will be installed:
libboost-dev libboost1.65-dev
0 upgraded, 2 newly installed, 0 to remove and 83 not upgraded.
Need to get 7,221 kB of archives.
After this operation, 119 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ca.archive.ubuntu.com/ubuntu bionic/main amd64 libboost1.65-dev amd64 1.65.1+dfsg-0ubuntu5 [7,218 kB]
Get:2 http://ca.archive.ubuntu.com/ubuntu bionic/main amd64 libboost-dev amd64 1.65.1.0ubuntu1 [3,128 B]
Fetched 7,221 kB in 7s (990 kB/s)
Selecting previously unselected package libboost1.65-dev:amd64.
(Reading database ... 250043 files and directories currently installed.)
Preparing to unpack .../libboost1.65-dev_1.65.1+dfsg-0ubuntu5_amd64.deb ...
Unpacking libboost1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...
Selecting previously unselected package libboost-dev:amd64.
Preparing to unpack .../libboost-dev_1.65.1.0ubuntu1_amd64.deb ...
Unpacking libboost-dev:amd64 (1.65.1.0ubuntu1) ...
Setting up libboost1.65-dev:amd64 (1.65.1+dfsg-0ubuntu5) ...
Setting up libboost-dev:amd64 (1.65.1.0ubuntu1) ...

(3) Then I installed those suggested packages, and create symbol links:

syu@VAN-2000-NEW:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_filesystem.so.1.65.1 libboost_filesystem.so
syu@VAN-2000-NEW:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_system.so.1.65.1 libboost_system.so
syu@VAN-2000-NEW:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_thread.so.1.65.1 libboost_thread.so

otherwise, I got link errors:

syu@VAN-201425-NEW:~/ml/caffe$ make
LD -o .build_release/lib/libcaffe.so.1.0.0
/usr/bin/x86_64-linux-gnu-ld: cannot find -lboost_system
/usr/bin/x86_64-linux-gnu-ld: cannot find -lboost_filesystem
/usr/bin/x86_64-linux-gnu-ld: cannot find -lboost_thread
collect2: error: ld returned 1 exit status
Makefile:582: recipe for target '.build_release/lib/libcaffe.so.1.0.0' failed
make: *** [.build_release/lib/libcaffe.so.1.0.0] Error 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment