Install DIGITS and NVcaffe on CentOS 7

install-digits-centos7.sh

# nvidia-docker run -ti nvidia/cuda:7.0-devel-centos7

yum update
yum install epel-release
yum install gcc gcc-c++ git vim python-devel python-pip make cmake \
    blas-devel lapack-devel openblas-devel \
    protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel libjpeg-devel libpng-devel

# Too old:
# yum install Cython scipy python-matplotlib
yum install numpy python-pandas python-imaging protobuf-python h5py
yum-builddep python-matplotlib

cd ~
git clone https://github.com/NVIDIA/caffe.git
cd caffe
cat python/requirements.txt | xargs -n1 pip install
mkdir build
cd build
cmake .. -DBLAS=Open
make -j12

# DIGITS
yum install python-gevent python-flask python-flask-wtf
cd ~
git clone https://github.com/NVIDIA/digits.git
cd digits
cat requirements.txt | xargs -n1 pip install
export CAFFE_HOME=~/caffe
./digits-devserver

Hi, does anybody know whether this works as well for earlier Centos versions, e.g. 6.5? And, if not, how the script must be modified to make it work? Thanks, D-E

I'm install in centos 7, but I not had success:
collect2: error: ld returned 1 exit status
make[2]: *** [tools/train_net] Error 1
make[2]: *** [tools/compute_image_mean] Error 1
make[1]: *** [tools/CMakeFiles/train_net.dir/all] Error 2
make[1]: *** [tools/CMakeFiles/compute_image_mean.dir/all] Error 2
/bin/ld: warning: libhdf5.so.8.0.2, needed by /opt/cmb/lib/libhdf5_hl.so.1.8.13, not found (try using -rpath or -rpath-link)
/bin/ld: warning: libsz.so.2, needed by /opt/cmb/lib/libhdf5_hl.so.1.8.13, not found (try using -rpath or -rpath-link)
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Pset_fapl_core' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5P_CLS_FILE_ACCESS_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_cset' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Dread'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dget_create_plist' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pclose'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tarray_create2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tget_member_name'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_order' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Sget_simple_extent_ndims'
../lib/libcaffe-nv.so.0.15.13: undefined reference to cv::imread(cv::String const&, int)' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_SCHAR_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Eset_auto2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Screate_simple'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Aget_type' ../lib/libcaffe-nv.so.0.15.13: undefined reference toH5Fcreate'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_nmembers' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tconvert'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_I8BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Eset_auto1'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_UCHAR_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pset'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_UINT_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Screate'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Eget_auto1' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Eauto_is_v2'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_size' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tget_strpad'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Oclose' ../lib/libcaffe-nv.so.0.15.13: undefined reference tocv::String::allocate(unsigned long)'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5P_CLS_DATASET_XFER_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tcreate'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_LONG_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tenum_insert'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Pset_chunk' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_IEEE_F64LE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Sselect_hyperslab' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Dwrite'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_REF_OBJ_g' ../lib/libcaffe-nv.so.0.15.13: undefined reference tocv::String::deallocate()'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_I16LE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tset_tag'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dset_extent' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pget_chunk'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Iregister' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tvlen_create'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_I8LE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Awrite'
../lib/libcaffe-nv.so.0.15.13: undefined reference to cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tset_cset'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Fclose' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5free_memory'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U64BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_LDOUBLE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_super' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tis_variable_str'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_native_type' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tinsert'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U64LE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pset_file_image_callbacks'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tequal' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tset_strpad'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Gclose' ../lib/libcaffe-nv.so.0.15.13: undefined reference toH5check_version'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U16LE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Oopen'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Lget_name_by_idx' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tset_order'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_I64LE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tenum_create'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Aopen' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Dget_space'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Lexists' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5P_CLS_DATASET_CREATE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_array_dims2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tget_class'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_sign' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_IEEE_F32LE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_INT_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_C_S1_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Sget_simple_extent_npoints' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_STD_I16BE_g'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Gcreate2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Oexists_by_name'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_member_offset' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_FORTRAN_S1_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_member_value' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_STD_U32LE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Iregister_type' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Aget_space'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tcopy' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_STD_I64BE_g'
../lib/libcaffe-nv.so.0.15.13: undefined reference to cv::error(int, cv::String const&, char const*, char const*, int)' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tclose'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_IEEE_F32BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Idestroy_type'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_ULLONG_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Iget_type'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Rcreate' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_LLONG_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Sclose' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_SHORT_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dopen2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Dget_type'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_DOUBLE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Iobject_verify'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dvlen_reclaim' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Aread'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dclose' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5open'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_NATIVE_FLOAT_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Oget_info'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Iremove_verify' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Ldelete'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U16BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tget_tag'
../lib/libcaffe-nv.so.0.15.13: undefined reference to H5Gget_info' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toHDfprintf'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Literate' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_USHORT_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Pcopy' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Sget_simple_extent_dims'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Lmove' ../lib/libcaffe-nv.so.0.15.13: undefined reference toH5Gopen2'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Dcreate2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_IEEE_F64BE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Pset_deflate' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pcreate'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Eget_auto2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pset_file_image'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_I32BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_STD_U8LE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Pset_preserve' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Pset_fill_value'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Tget_array_ndims' ../lib/libcaffe-nv.so.0.15.13: undefined reference toH5Fopen'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U8BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_STD_I32LE_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Adelete' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Aiterate2'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Aclose' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Rdereference'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5T_STD_U32BE_g' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5Tset_size'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to H5Acreate2' /opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference toH5T_NATIVE_ULONG_g'
/opt/cmb/lib/libhdf5_hl.so.1.8.13: undefined reference to `H5Tget_member_type'
collect2: error: ld returned 1 exit status
make[2]: *** [tools/net_speed_benchmark] Error 1
make[1]: *** [tools/CMakeFiles/net_speed_benchmark.dir/all] Error 2
/bin/ld: warning: libhdf5.so.8.0.2, needed by /opt/cmb/lib/libhdf5_hl.so.1.8.13, not found (try using -rpath or -rpath-link)
/bin/ld: warning: libsz.so.2, needed by /opt/cmb/lib/libhdf5_hl.so.1.8.13, not found (try using -rpath or -rpath-link)

Help.

It is very helpful, thanks!

Hi Luke,

Looks like the epel release is missing the python-pip package and a few others. I've been browsing the web but they all seem to point to epel for these, but no joy. Any suggestions?
sudo yum update
Loaded plugins: aliases, changelog, fastestmirror, kabi, langpacks, product-id, search-disabled-repos, subscription-manager,
: tmprepo, verify, versionlock
This system is not registered with Subscription Management. You can use subscription-manager to register.
Loading support for Red Hat kernel ABI
Loading mirror speeds from cached hostfile

• base: centos.pymesolutionsweb.com
• extras: mirrors.kernel.org
• updates: mirror.hmc.edu
  No packages marked for update
  ekoch@bil9001.nvidia.com> sudo yum install epel-release
  Loaded plugins: aliases, changelog, fastestmirror, kabi, langpacks, product-id, search-disabled-repos, subscription-manager,
  : tmprepo, verify, versionlock
  This system is not registered with Subscription Management. You can use subscription-manager to register.
  Loading support for Red Hat kernel ABI
  Loading mirror speeds from cached hostfile
• base: centos.pymesolutionsweb.com
• extras: mirrors.kernel.org
• updates: mirror.hmc.edu
  Package epel-release-7-9.noarch already installed and latest version
  Nothing to do
  ekoch@bil9001.nvidia.com> sudo yum install python-pip
  Loaded plugins: aliases, changelog, fastestmirror, kabi, langpacks, product-id, search-disabled-repos, subscription-manager,
  : tmprepo, verify, versionlock
  This system is not registered with Subscription Management. You can use subscription-manager to register.
  Loading support for Red Hat kernel ABI
  Loading mirror speeds from cached hostfile
• base: centos.pymesolutionsweb.com
• extras: mirrors.kernel.org
• updates: mirror.hmc.edu
  No package python-pip available.
  Error: Nothing to do

Luke, nm as I thought this was a repository issue. Will inform LA about it for his centos 7 image.

just fyi other readers i used this repository to fix my python-pip issue:
cat /etc/yum.repos.d/epel.repo
[epel]
name=Extra Packages for Enterprise Linux 7 - $basearch
baseurl=http://dl.fedoraproject.org/pub/epel/7/$basearch
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=$basearch
failovermethod=priority
enabled=1
gpgcheck=1
gpgkey=http://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7

[epel-debuginfo]
name=Extra Packages for Enterprise Linux 7 - $basearch - Debug
baseurl=http://dl.fedoraproject.org/pub/epel/7/$basearch/debug
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=epel-debug-7&arch=$basearch
failovermethod=priority
enabled=1
gpgkey=http://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-7
gpgcheck=1

[epel-source]

Found a couple of errors when I installed this. With some minor intermediate edits installation does go okay. here are the errors i get...

1. incorrect declarations:
   [ 31%] Building CXX object src/caffe/CMakeFiles/caffe.dir/layers/euclidean_loss_layer.cpp.o
   In file included from /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp:3:0:
   /root/caffe/include/caffe/layers/detectnet_transform_layer.hpp: In member function 'bool caffe::AugmentSelection::doDesaturation() const':
   /root/caffe/include/caffe/layers/detectnet_transform_layer.hpp:38:64: error: 'UINT8_MAX' was not declared in this scope
   bool doDesaturation() const { return saturation < (1.0 - 1.0/UINT8_MAX); }
   ^
   [ 33%] Building CXX object src/caffe/CMakeFiles/caffe.dir/layers/exp_layer.cpp.o
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp: In member function 'void caffe::DetectNetTransformationLayer::retrieveMeanImage(cv::Size)':
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp:86:23: error: 'UINT8_MAX' was not declared in this scope
   data_mean_ /= Dtype(UINT8_MAX);
   ^
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp: In member function 'void caffe::DetectNetTransformationLayer::retrieveMeanChannels()':
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp:94:56: error: 'UINT8_MAX' was not declared in this scope
   mean_values_.fill(t_param_.mean_value(0) / Dtype(UINT8_MAX));
   ^
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp: In member function 'caffe::DetectNetTransformationLayer::Mat3v caffe::DetectNetTransformationLayer::transform_image_cpu(const Mat3v&, const caffe::AugmentSelection&)':
   /root/caffe/src/caffe/layers/detectnet_transform_layer.cpp:420:33: error: 'UINT8_MAX' was not declared in this scope
   Mat3v img = src_img.clone() / UINT8_MAX;
   ^
   [ 33%] Building CXX object src/caffe/CMakeFiles/caffe.dir/layers/filter_layer.cpp.o

2. Fails for existing directories during clone process:

The procedure cannot do the cloning if /root/caffe and /root/digits already exist. My hack was to remove those directories before the cloning steps. But i need to comment out the cloning in order to do the hacks to fix the step 1 above....

darn i lost my hacks. for sure the declaration issues are in these files:
find /root/caffe -type f -exec grep -l UINT8_MAX {} ; | & tee found_unit.log
/root/caffe/include/caffe/layers/detectnet_transform_layer.hpp
/root/caffe/src/caffe/layers/detectnet_transform_layer.cpp
/root/caffe/src/caffe/layers/detectnet_transform_layer.cu

I'll try to redo them.... if you can get it before i do please let me know.

thanks
Elliot

Luke, here are my proposed changes to the Centos 7 installation. Please be so kind as to review and advise. If you agree, I would like to see these fixes or whatever you think is better into the mainstream code.

still trying to figure out how the pull request mechanism works, so here is the link in the meantime:

https://github.com/elliotekoch/Fixes-for-Digits-installation-on-Centos-7

Thanks,
Elliot