caffe_installation_notes.txt

==============================================================================
Caffe Installation Notes

[2016-Nov-21] Arch Linux x86_64 | Intel Core i7-4790 CPU | ...
==============================================================================

* Caffe home page: http://caffe.berkeleyvision.org/installation.html
* GitHub: https://github.com/BVLC/caffe
* Caffe Users Group (ask questions here, not GitHub issues!): https://groups.google.com/forum/#!forum/caffe-users

------------------------------------------------------------------------------

[2016-Nov-21] Installed Caffe 1.0.0-rc3 in p2 (py27 | Python 2.7) venv

DEPENDENCIES:
=============

openblas (installed):

    (py27) [victoria@victoria caffe]$ pacman -Ss openblas

        archlinuxfr/openblas 0.2.19-1 [installed]

boost (installed):

    (py27) [victoria@victoria caffe]$ pacman -Ss boost

        extra/boost 1.62.0-3 [installed]
        extra/boost-libs 1.62.0-3 [installed]
        community/websocketpp 0.7.0-1

opencv (installed):

    (py27) [victoria@victoria caffe]$ pacman -Ss opencv

        extra/opencv 3.1.0-3
        extra/opencv-samples 3.1.0-3

    (py27) [victoria@victoria caffe]$ P

          [P: python]
        Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40)
        [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
        Type "help", "copyright", "credits" or "license" for more information.
        Anaconda is brought to you by Continuum Analytics.
        Please check out: http://continuum.io/thanks and https://anaconda.org

        >>> import cv2
        >>> cv2.__version__
            '3.1.0'

        >>> [Ctrl-D]

    (py27) [victoria@victoria caffe]$

------------------------------------------------------------------------------
# PLAN "A":
===========

* I first tried compiling caffe using cmake (reference above) per the Caffe
website instructions, but it failed (multiple errors):

    http://caffe.berkeleyvision.org/installation.html

    CMake Build:
    ------------

    In lieu of manually editing Makefile.config to configure the build,
    Caffe offers an unofficial CMake build thanks to @Nerei, @akosiorek,
    and other members of the community. It requires CMake version >= 2.8.7.
    The basic steps are as follows:

        mkdir build
        cd build
        cmake ..
        make all
        make install
        make runtest

        See PR #1667 for options and details:
        https://github.com/BVLC/caffe/pull/1667

Chief among those errors was a "clapack"-related error(s) ...; I could have
done "pacaur -S atlas-clapack", BUT that conflicts with/wanted to uninstall
OpenBLAS (which I prefer and use with other applications)!

------------------------------------------------------------------------------
# PLAN "B":
===========

... So I went with the more involved (but more configurable) "make"
compilation/installation:

I first copied

    /mnt/Vancouver/apps/caffe/Makefile.config.example

to

    /mnt/Vancouver/apps/caffe/Makefile.config

and made edits to that file.

Aside: this process, though ultimately successful, was very buggy:
took ~4-5 hrs!!

I needed to multiply try to install (make) caffe, and then pycaffe (both
are needed!), in an iterative process:

    delete the caffe folder;
    reclone (git clone https://github.com/BVLC/caffe);
    tweak the "Makefile.config";
    repeat the installation

... until ultimate success.  Cumbersome, but (for me) the simplest,
error-free approach.

I made sure to save a copy of the "Makefile.config" file, replacing it in
the newly-recloned caffe repo at each (re)start.

This is the "Makefile.config" file that ultimately worked (superfluous
lines removed):

    ------------------------------------------------------------------------------
    ## /mnt/Vancouver/apps/caffe/Makefile.config
    CPU_ONLY := 1
    OPENCV_VERSION := 3
    CUSTOM_CXX := g++
    BLAS := open
    BLAS_INCLUDE := /home/victoria/anaconda3/envs/py27/lib/
    BLAS_LIB := /home/victoria/anaconda3/envs/py27/lib/
    ANACONDA_HOME := $(HOME)/anaconda3

    PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
        $(ANACONDA_HOME)/include/python2.7 \
        $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
        /home/victoria/anaconda3/envs/py27/include/python2.7/ \
        /home/victoria/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/core/include/
    PYTHON_LIB := $(ANACONDA_HOME)/lib
    INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
    LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
    BUILD_DIR := build
    DISTRIBUTE_DIR := distribute
    TEST_GPUID := 0
    Q ?= @
    ------------------------------------------------------------------------------

MAKEFILE-RELATED ERRORS/NOTES:
==============================

    ----------------------------------------
    lopenblas:
        # (py27) [victoria@victoria caffe]$ find 2>/dev/null /home/victoria/anaconda3/envs/py27/ -iname "*libopenblas.so"
        #     /home/victoria/anaconda3/envs/py27/lib/libopenblas.so
        BLAS_INCLUDE := /home/victoria/anaconda3/envs/py27/lib/
        BLAS_LIB := /home/victoria/anaconda3/envs/py27/lib/
    ----------------------------------------
    ANACONDA_HOME := $(HOME)/anaconda3
        not:
    ANACONDA_HOME := $(HOME)/anaconda
    ----------------------------------------
    # Python.h:
    # python/caffe/_caffe.cpp:1:52: fatal error: Python.h: No such file or directory  #include <Python.h>  // NOLINT(build/include_alpha)
    # https://groups.google.com/forum/#!topic/caffe-users/2pvE3NgcXJo
    # (py27) [victoria@victoria caffe]$ find 2>/dev/null /home/victoria/anaconda3/envs/py27/ -iname "*python.h*"
    # /home/victoria/anaconda3/envs/py27/include/python2.7/Python.h
    # python/caffe/_caffe.cpp:10:31: fatal error: numpy/arrayobject.h: No such file or directory
    # ... find Anaconda numpy install ...
    # (py27) [victoria@victoria caffe]$ find 2>/dev/null /home/victoria/anaconda3/envs/py27/ -iname "*numpy"
    # want this one:
    # /home/victoria/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/core/include/numpy
    PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
        $(ANACONDA_HOME)/include/python2.7 \
        $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
        /home/victoria/anaconda3/envs/py27/include/python2.7/ \
        /home/victoria/anaconda3/envs/py27/lib/python2.7/site-packages/numpy/core/include/


INSTALLATION:
=============

NOTE: p2 venv (py27 | Python 2.7 -- preferred by Caffe)!

    cd /mnt/Vancouver/apps
    git clone https://github.com/BVLC/caffe
    cd caffe
    make all -j8

    export PYTHONPATH=${PYTHONPATH}:/mnt/Vancouver/apps/caffe/python/
    make pycaffe

    ## Python: ImportError: No module named google.protobuf.internal
    ## https://groups.google.com/forum/#!topic/caffe-users/9Q10WkpCGxs :

    pip install protobuf

    ...
    SUCCESS!!  :-D  Woot!!

At each failed install:
-----------------------

    save backup copy of Makefile.config   ## /mnt/Vancouver/Makefile.config
    cd ..
    rm -fR caffe                          ## /mnt/Vancouver/apps/caffe
    git clone https://github.com/BVLC/caffe
    cd caffe
    cp /mnt/Vancouver/Makefile.config /mnt/Vancouver/apps/caffe
    make all -j8
    ...

Once Caffe was successfully installed, I added this $PYTHONPATH to my
~/.bashrc:

    export PYTHONPATH=/mnt/Vancouver/apps/caffe/python:$PYTHONPATH

(py27) [victoria@victoria caffe]$ P

      [P: python]
    Python 2.7.12 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:42:40)
    [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    Anaconda is brought to you by Continuum Analytics.
    Please check out: http://continuum.io/thanks and https://anaconda.org

    >>> import caffe
    >>> caffe.__version__
        '1.0.0-rc3'
    >>> [Ctrl-D]

(py27) [victoria@victoria caffe]$

Q.E.D.  :-D