System Requirements For Running Caffe
The GPU-enabled version of Caffe has the following requirements:
64-bit Linux (This guide is written for Ubuntu 14.04)
NVIDIA® CUDA® 7.5 (CUDA 8.0 required for NVIDIA Pascal™ GPUs)
cuDNN v5.1
You'll also need an NVIDIA GPU supporting compute capability 3.0 or higher.
How to Download and Install Caffe Step 1. Install CUDA
Do this with your system's package manager.
To use Caffe with NVIDIA GPUs, the first step is to install the CUDA Toolkit.
Step 2. Install cuDNN:
Once the CUDA Toolkit is installed, download cuDNN v5.1 Library for Linux (note that you'll need to register for the Accelerated Computing Developer Program).
Once downloaded, uncompress the files and copy them into the CUDA Toolkit directory (assumed here to be in /usr/local/cuda/):
$ sudo tar -xvf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local
Step 3. Install Dependencies:
Caffe depends on several libraries that should be available from your system's package manager.
On Ubuntu 14.04+, the following commands will install the necessary libraries:
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler libgflags-dev libgoogle-glog-dev liblmdb-dev libatlas-base-dev git
$ sudo apt-get install --no-install-recommends libboost-all-dev
Step 4. Install NCCL:
NVIDIA NCCL is required to run Caffe on more than one GPU. NCCL can be installed with the following commands:
$ git clone https://github.com/NVIDIA/nccl.git
$ cd nccl
$ sudo make install -j$(nproc)
NCCL libraries and headers will be installed in /usr/local/lib and /usr/local/include.
Step 5. Install Caffe:
We recommend installing the latest released version of Caffe from NVIDIA's fork, found at https://github.com/NVIDIA/caffe/releases. As of this writing, the latest released version is 0.15.9.
$ wget https://github.com/NVIDIA/caffe/archive/v0.15.9.tar.gz
$ tar -zxf v0.15.9.tar.gz
$ cd caffe-0.15.9
$ cp Makefile.config.example Makefile.config
Open the newly created Makefile.config in a text editor and make the following changes:
Uncomment the line USE_CUDNN := 1. This enables cuDNN acceleration.
Uncomment the line USE_NCCL := 1. This enables NCCL which is required to run Caffe on multiple GPUs.
You'll also need to make the following changes to Makefile.config to address a build issue noted in Ubuntu 16.04 where the hdf5.h header could not be located:
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/
Save and close the file. You're now ready to compile Caffe.
$ time make all -j$(nproc)
When this command completes, the Caffe binary will be available at build/tools/caffe.
You may now copy (or move) your installation away from the build location to one you desire.
My example: I moved the caffe build artifacts into /apps/caffe/nvidia-blobs , renamed the tools subdirectory to bin and wrote an environment module file to load it when needed:
sudo mkdir -p /apps/caffe/nvidia-blob
sudo cp -vr build/* /apps/caffe/nvidia-blob/
ls -al /apps/caffe/nvidia-blob/
ls -al /apps/caffe/nvidia-blob/tools/
sudo mv /apps/caffe/nvidia-blob/tools /apps/caffe/nvidia-blob/bin
Then I tried running the application as installed:
/apps/caffe/nvidia-blob/bin/caffe.bin
caffe.bin: command line brew
usage: caffe <command> <args>
commands:
train train or finetune a model
test score a model
device_query show GPU diagnostic information
time benchmark model execution time
Flags from tools/caffe.cpp:
-gpu (Optional; run in GPU mode on given device IDs separated by ','.Use
'-gpu all' to run on all available GPUs. The effective training batch
size is multiplied by the number of devices.) type: string default: ""
-iterations (The number of iterations to run.) type: int32 default: 50
-model (The model definition protocol buffer text file.) type: string
default: ""
-sighup_effect (Optional; action to take when a SIGHUP signal is received:
snapshot, stop or none.) type: string default: "snapshot"
-sigint_effect (Optional; action to take when a SIGINT signal is received:
snapshot, stop or none.) type: string default: "stop"
-snapshot (Optional; the snapshot solver state to resume training.)
type: string default: ""
-solver (The solver definition protocol buffer text file.) type: string
default: ""
-weights (Optional; the pretrained weights to initialize finetuning,
separated by ','. Cannot be set simultaneously with snapshot.)
type: string default: ""
Then I checked the application version:
/apps/caffe/nvidia-blob/bin/caffe --version
caffe version 0.15.14
Running ldd against the new binary:
ldd /apps/caffe/nvidia-blob/bin/caffe
linux-vdso.so.1 => (0x00007ffc5a790000)
libcaffe-nv.so.0.15 => /apps/caffe/nvidia-blob/bin/../lib/libcaffe-nv.so.0.15 (0x00007f63a3cad000)
libcudart.so.8.0 => /usr/local/cuda-8.0/lib64/libcudart.so.8.0 (0x00007f63a3a47000)
libglog.so.0 => /usr/lib/x86_64-linux-gnu/libglog.so.0 (0x00007f63a37e5000)
libgflags.so.2 => /usr/lib/x86_64-linux-gnu/libgflags.so.2 (0x00007f63a35c4000)
libboost_system.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_system.so.1.58.0 (0x00007f63a33c0000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f63a303d000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f63a2e27000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f63a2a5e000)
libcublas.so.8.0 => /usr/local/cuda-8.0/lib64/libcublas.so.8.0 (0x00007f63a00ad000)
libcurand.so.8.0 => /usr/local/cuda-8.0/lib64/libcurand.so.8.0 (0x00007f639c144000)
libprotobuf.so.9 => /usr/lib/x86_64-linux-gnu/libprotobuf.so.9 (0x00007f639be26000)
libhdf5_serial_hl.so.10 => /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.10 (0x00007f639bc05000)
libhdf5_serial.so.10 => /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.10 (0x00007f639b768000)
libleveldb.so.1 => /usr/lib/x86_64-linux-gnu/libleveldb.so.1 (0x00007f639b50e000)
liblmdb.so.0 => /usr/lib/x86_64-linux-gnu/liblmdb.so.0 (0x00007f639b2f8000)
libopencv_core.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_core.so.2.4 (0x00007f639aece000)
libopencv_highgui.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_highgui.so.2.4 (0x00007f639ac85000)
libopencv_imgproc.so.2.4 => /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.2.4 (0x00007f639a7f9000)
libboost_thread.so.1.58.0 => /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0 (0x00007f639a5d3000)
libcudnn.so.5 => /usr/local/cuda-8.0/lib64/libcudnn.so.5 (0x00007f63957ff000)
libnccl.so.1 => /usr/local/lib/libnccl.so.1 (0x00007f6393fa4000)
libcblas.so.3 => /usr/lib/libcblas.so.3 (0x00007f6393d82000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6393a79000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f639385b000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6393657000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f639344e000)
libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007f6393233000)
/lib64/ld-linux-x86-64.so.2 (0x000055dca7fec000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6393018000)
libsz.so.2 => /usr/lib/x86_64-linux-gnu/libsz.so.2 (0x00007f6392e15000)
libsnappy.so.1 => /usr/lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007f6392c0d000)
libGL.so.1 => /usr/lib/nvidia-367/libGL.so.1 (0x00007f639297d000)
libtbb.so.2 => /usr/lib/x86_64-linux-gnu/libtbb.so.2 (0x00007f6392740000)
libjpeg.so.8 => /usr/lib/x86_64-linux-gnu/libjpeg.so.8 (0x00007f63924e6000)
libpng12.so.0 => /lib/x86_64-linux-gnu/libpng12.so.0 (0x00007f63922c1000)
libtiff.so.5 => /usr/lib/x86_64-linux-gnu/libtiff.so.5 (0x00007f639204e000)
libjasper.so.1 => /usr/lib/x86_64-linux-gnu/libjasper.so.1 (0x00007f6391df7000)
libIlmImf-2_2.so.22 => /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22 (0x00007f6391928000)
libHalf.so.12 => /usr/lib/x86_64-linux-gnu/libHalf.so.12 (0x00007f63916e5000)
libgtk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0 (0x00007f639109a000)
libgdk-x11-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk-x11-2.0.so.0 (0x00007f6390de4000)
libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f6390b91000)
libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f6390880000)
libgtkglext-x11-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libgtkglext-x11-1.0.so.0 (0x00007f639067b000)
libgdkglext-x11-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdkglext-x11-1.0.so.0 (0x00007f6390417000)
libdc1394.so.22 => /usr/lib/x86_64-linux-gnu/libdc1394.so.22 (0x00007f63901a1000)
libv4l1.so.0 => /usr/lib/x86_64-linux-gnu/libv4l1.so.0 (0x00007f638ff9a000)
libavcodec-ffmpeg.so.56 => /usr/lib/x86_64-linux-gnu/libavcodec-ffmpeg.so.56 (0x00007f638eb6e000)
libavformat-ffmpeg.so.56 => /usr/lib/x86_64-linux-gnu/libavformat-ffmpeg.so.56 (0x00007f638e771000)
libavutil-ffmpeg.so.54 => /usr/lib/x86_64-linux-gnu/libavutil-ffmpeg.so.54 (0x00007f638e501000)
libswscale-ffmpeg.so.3 => /usr/lib/x86_64-linux-gnu/libswscale-ffmpeg.so.3 (0x00007f638e272000)
libatlas.so.3 => /usr/lib/libatlas.so.3 (0x00007f638dcd3000)
libgfortran.so.3 => /usr/lib/x86_64-linux-gnu/libgfortran.so.3 (0x00007f638d9a8000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f638d785000)
libaec.so.0 => /usr/lib/x86_64-linux-gnu/libaec.so.0 (0x00007f638d57d000)
libGLX.so.0 => /usr/lib/nvidia-367/libGLX.so.0 (0x00007f638d34c000)
libGLdispatch.so.0 => /usr/lib/nvidia-367/libGLdispatch.so.0 (0x00007f638d063000)
libjbig.so.0 => /usr/lib/x86_64-linux-gnu/libjbig.so.0 (0x00007f638ce54000)
libIex-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12 (0x00007f638cc36000)
libIlmThread-2_2.so.12 => /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12 (0x00007f638ca2e000)
libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f638c82a000)
libpangocairo-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0 (0x00007f638c61d000)
libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f638c2e2000)
libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007f638c0dc000)
libatk-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libatk-1.0.so.0 (0x00007f638beb7000)
libcairo.so.2 => /usr/lib/x86_64-linux-gnu/libcairo.so.2 (0x00007f638bba2000)
libgdk_pixbuf-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x00007f638b980000)
libgio-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0 (0x00007f638b5f8000)
libpangoft2-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0 (0x00007f638b3e1000)
libpango-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0 (0x00007f638b195000)
libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007f638af52000)
libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007f638ad47000)
libXinerama.so.1 => /usr/lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007f638ab44000)
libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007f638a933000)
libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007f638a728000)
libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007f638a51e000)
libXcomposite.so.1 => /usr/lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007f638a31b000)
libXdamage.so.1 => /usr/lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007f638a117000)
libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f6389f05000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f6389cfc000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6389a8c000)
libGLU.so.1 => /usr/lib/x86_64-linux-gnu/libGLU.so.1 (0x00007f638981d000)
libXmu.so.6 => /usr/lib/x86_64-linux-gnu/libXmu.so.6 (0x00007f6389603000)
libpangox-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libpangox-1.0.so.0 (0x00007f63893e3000)
libraw1394.so.11 => /usr/lib/x86_64-linux-gnu/libraw1394.so.11 (0x00007f63891d4000)
libusb-1.0.so.0 => /lib/x86_64-linux-gnu/libusb-1.0.so.0 (0x00007f6388fbb000)
libv4l2.so.0 => /usr/lib/x86_64-linux-gnu/libv4l2.so.0 (0x00007f6388dad000)
libswresample-ffmpeg.so.1 => /usr/lib/x86_64-linux-gnu/libswresample-ffmpeg.so.1 (0x00007f6388b90000)
libva.so.1 => /usr/lib/x86_64-linux-gnu/libva.so.1 (0x00007f6388974000)
libzvbi.so.0 => /usr/lib/x86_64-linux-gnu/libzvbi.so.0 (0x00007f63886e8000)
libxvidcore.so.4 => /usr/lib/x86_64-linux-gnu/libxvidcore.so.4 (0x00007f63883d4000)
libx265.so.79 => /usr/lib/x86_64-linux-gnu/libx265.so.79 (0x00007f63877b5000)
libx264.so.148 => /usr/lib/x86_64-linux-gnu/libx264.so.148 (0x00007f6387410000)
libwebp.so.5 => /usr/lib/x86_64-linux-gnu/libwebp.so.5 (0x00007f63871b4000)
libwavpack.so.1 => /usr/lib/x86_64-linux-gnu/libwavpack.so.1 (0x00007f6386f8b000)
libvpx.so.3 => /usr/lib/x86_64-linux-gnu/libvpx.so.3 (0x00007f6386b66000)
libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007f63868bd000)
libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007f6386691000)
libtwolame.so.0 => /usr/lib/x86_64-linux-gnu/libtwolame.so.0 (0x00007f638646d000)
libtheoraenc.so.1 => /usr/lib/x86_64-linux-gnu/libtheoraenc.so.1 (0x00007f638622e000)
libtheoradec.so.1 => /usr/lib/x86_64-linux-gnu/libtheoradec.so.1 (0x00007f6386014000)
libspeex.so.1 => /usr/lib/x86_64-linux-gnu/libspeex.so.1 (0x00007f6385dfa000)
libshine.so.3 => /usr/lib/x86_64-linux-gnu/libshine.so.3 (0x00007f6385bed000)
libschroedinger-1.0.so.0 => /usr/lib/x86_64-linux-gnu/libschroedinger-1.0.so.0 (0x00007f6385918000)
libopus.so.0 => /usr/lib/x86_64-linux-gnu/libopus.so.0 (0x00007f63856cd000)
libopenjpeg.so.5 => /usr/lib/x86_64-linux-gnu/libopenjpeg.so.5 (0x00007f63854aa000)
libmp3lame.so.0 => /usr/lib/x86_64-linux-gnu/libmp3lame.so.0 (0x00007f6385235000)
libgsm.so.1 => /usr/lib/x86_64-linux-gnu/libgsm.so.1 (0x00007f6385026000)
libcrystalhd.so.3 => /usr/lib/x86_64-linux-gnu/libcrystalhd.so.3 (0x00007f6384e0b000)
libssh-gcrypt.so.4 => /usr/lib/x86_64-linux-gnu/libssh-gcrypt.so.4 (0x00007f6384bc1000)
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f63849a5000)
libmodplug.so.1 => /usr/lib/x86_64-linux-gnu/libmodplug.so.1 (0x00007f638461a000)
libgme.so.0 => /usr/lib/x86_64-linux-gnu/libgme.so.0 (0x00007f63843cb000)
libbluray.so.1 => /usr/lib/x86_64-linux-gnu/libbluray.so.1 (0x00007f6384182000)
libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f6383e52000)
libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007f6383c41000)
libquadmath.so.0 => /usr/lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007f6383a02000)
libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007f6383757000)
libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007f6383535000)
libpixman-1.so.0 => /usr/lib/x86_64-linux-gnu/libpixman-1.so.0 (0x00007f638328c000)
libxcb-shm.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007f6383088000)
libxcb-render.so.0 => /usr/lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007f6382e7e000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f6382c5b000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f6382a40000)
libharfbuzz.so.0 => /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0 (0x00007f63827e2000)
libthai.so.0 => /usr/lib/x86_64-linux-gnu/libthai.so.0 (0x00007f63825d8000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f63823af000)
libXt.so.6 => /usr/lib/x86_64-linux-gnu/libXt.so.6 (0x00007f6382145000)
libudev.so.1 => /lib/x86_64-linux-gnu/libudev.so.1 (0x00007f6382125000)
libv4lconvert.so.0 => /usr/lib/x86_64-linux-gnu/libv4lconvert.so.0 (0x00007f6381eab000)
libsoxr.so.0 => /usr/lib/x86_64-linux-gnu/libsoxr.so.0 (0x00007f6381c45000)
libnuma.so.1 => /usr/lib/x86_64-linux-gnu/libnuma.so.1 (0x00007f6381a3a000)
libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007f6381830000)
liborc-0.4.so.0 => /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007f63815b0000)
libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007f63812ce000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f6381084000)
libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f6380e51000)
libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f6380c1a000)
libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f638099a000)
libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f63805e0000)
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f638037b000)
libidn.so.11 => /usr/lib/x86_64-linux-gnu/libidn.so.11 (0x00007f6380148000)
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f637ff35000)
libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007f637fd30000)
libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007f637fb2a000)
libgraphite2.so.3 => /usr/lib/x86_64-linux-gnu/libgraphite2.so.3 (0x00007f637f904000)
libdatrie.so.1 => /usr/lib/x86_64-linux-gnu/libdatrie.so.1 (0x00007f637f6fc000)
libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6 (0x00007f637f4f4000)
libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6 (0x00007f637f2d9000)
libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007f637f0b7000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f637eea2000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f637ebd0000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f637e9a1000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f637e79c000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f637e591000)
libicuuc.so.55 => /usr/lib/x86_64-linux-gnu/libicuuc.so.55 (0x00007f637e1fd000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f637dff7000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f637ddf3000)
libicudata.so.55 => /usr/lib/x86_64-linux-gnu/libicudata.so.55 (0x00007f637c33b000)
To see the modules available on my system, I ran:
module avail
------------------------------------------------------------------------------------------- /usr/share/modules/versions --------------------------------------------------------------------------------------------
3.2.10
------------------------------------------------------------------------------------------ /usr/share/modules/modulefiles ------------------------------------------------------------------------------------------
caffe/nvidia-blob gcc/4.9.2 gcc/5.2.0 gcc-linaro-toolchain/6.1.1 module-git null use.own
dot gcc/4.9.3 gcc/5.3.0 llvm/3.7.0 module-info openmpi/1.10.1
ffmpeg/git gcc/5.1.0 gcc-linaro-toolchain/4.9 mkl/11.3.1.150 modules openmpi/1.10.2
Then created a new module subdirectory for caffe:
sudo mkdir -p /usr/share/modules/modulefiles/caffe/nvidia-blob
And wrote a module file for caffe, versioned as nvidia-blob on my system:
nano /usr/share/modules/modulefiles/caffe/nvidia-blob
Module content is shown below:
#%Module1.0#####################################################################
##
## caffe media transcoder modulefile
## By Dennis Mungai <dmngaie@gmail.com>
## December, 2016
##
# for Tcl script use only
set appname caffe
set version nvidia-blob
set prefix /apps/${appname}/${version}
set exec_prefix ${prefix}/bin
prepend-path PATH ${exec_prefix}
prepend-path LD_LIBRARY_PATH ${prefix}/lib
Then proceeded to load and test the module:
module load caffe
Let's see if caffe is on the system path:
which caffe
/apps/caffe/nvidia-blob/bin/caffe
Then we can now clear all modules by running:
module purge
Prepare an Image Database:
A database of images is required as input to test the training performance of Caffe. Caffe comes with models that are set up to use images from the ILSVRC12 challenge ("ImageNet"). The original image files can be downloaded from http://image-net.org/download-images (you'll need to make an account and agree to their terms). Once you've downloaded and unpacked the original image files onto your system, continue with the steps below. It's assumed that the original images are stored on your disk like:
/path/to/imagenet/train/n01440764/n01440764_10026.JPEG
/path/to/imagenet/val/ILSVRC2012_val_00000001.JPEG
Step 6. Download Auxiliary Data:
$ ./data/ilsvrc12/get_ilsvrc_aux.sh
Step 7. Create the Database:
Open the file examples/imagenet/create_imagenet.sh in a text editor and make the following changes:
Change the variables TRAIN_DATA_ROOT and VAL_DATA_ROOT to the path where you unpacked the original images.
Set RESIZE=true so the images will be resized properly before being added to the database.
Save and close the file. You're now ready to create the image databases with the following command:
$ ./examples/imagenet/create_imagenet.sh
Then, create the required image mean file with:
$ ./examples/imagenet/make_imagenet_mean.sh