Building TensorFlow from source

building_tensorflow.md

Building TensorFlow from source

The official instructions on building TensorFlow are here: https://www.tensorflow.org/install/install_sources

Prerequisites

We are assuming a build with CUDA support, as well as including SIMD optimizations (SSE3, SSE4, AVX, AVX2, FMA).

Good to know: The compute capabilities for

• Maxwell TITAN X: 5.2
• Pascal TITAN X (2016): 6.1

On new systems, one will have to install CUDA, CuDNN, plus the following dependencies:

$ sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel libcupti-dev

(Leave out libcupti-dev when not building with GPU support.)

Installing Bazel

Bazel is Google's own build system, required to build TensorFlow. Building TensorFlow usually requires an up-to-date version of Bazel; there is a good chance that whatever your package manager provides will be outdated.

There are various ways to obtain a build of Bazel (see https://bazel.build/versions/master/docs/install-ubuntu.html).

Option 1: Using the Bazel APT repository

Recommended by Google, but you need to be comfortable adding another APT package souce.

  $ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
  $ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
  $ sudo apt-get update && sudo apt-get install bazel

Option 2: Building from source

In case you prefer building from source, it's unfortunately not as easy as cloning the Git repository and typing make. Recent versions of Bazel can only be built with Bazel, unless one downloads a distribution source build, which contains some already pre-generated files. With one such installation in place, one could build Bazel straight from the repository source, but that's probably not necessary.

So we will go with building a distribution build, which is reasonably straightforward:

• Download a distribution package from the releases page. The current version at the time of writing was 0.5.3.

  $ mkdir bazel && cd bazel
  $ wget https://github.com/bazelbuild/bazel/releases/download/0.5.3/bazel-0.5.3-dist.zip
  

• Unzip the sources. This being a zip file, the files are stored without containing folder. Glad we already put it in its own directory...

  $ unzip bazel-0.5.3-dist.zip
  

• Compile Bazel

  $ bash ./compile.sh
  

• The output executable is now located in output/bazel. Add a PATH entry to your .bashrc, or just export it in your current shell:

  $ export PATH=`pwd`/output:$PATH
  

You should now be able to call the bazel executable from anywhere on your filesystem.

Installing TensorFlow

Notes on building with unsupported system

• Additional steps may be necessary when building TensorFlow on an officially unsupported system (e.g. on a non-LTS Ubuntu version). For example, Ubuntu versions >16.04 ship with GCC 6 as default compiler, but CUDA 8 (still) requires a GCC compiler from the GCC 5.x series. In this case:

  • Build GCC 5.x from source and install, preferably user-local. (https://github.com/kmhofmann/build_stuff provides a script to build various versions of GCC from source.)

    Add the respective paths to the PATH and LD_LIBRARY_PATH variables, e.g.:

    $ export PATH=$HOME/local/bin:$PATH
    $ export LD_LIBRARY_PATH=$HOME/local/lib64:$LD_LIBRARY_PATH
    

    Ensure that gcc --version is the desired version.

  • Proceed with building, as described below. In case of errors, you may need to apply #9482, either by checking out a version >1.1.0 below (once released), building straight from the master branch, or applying the patch manually (it's small).

Building TensorFlow

• Activate your respective Python 3 based virtual environment.

• Clone the sources, and check out the desired branch. At the time of writing, 1.3.0 was the latest version; adjust if necessary.

  $ git clone https://github.com/tensorflow/tensorflow
  $ cd tensorflow
  $ git checkout v1.3.0
  

• Run the configuration script

  $ ./configure
  

  You can leave most defaults, but do specify the following (or similar):

  CUDA support -> Y
  CUDA compute capability -> 5.2,6.1
  

• Compile TensorFlow using Bazel.

  I got the special build options from here: http://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions

  $ bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
  

  (Leave out --config=cuda when not building with GPU support.)

• We still need to build a Python package using the now generated build_pip_package script.

  $ bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
  

• Now there will be a package inside /tmp/tensorflow_pkg, which can be installed with pip.

  $ pip install /tmp/tensorflow_pkg/tensorflow-1.3.0-cp35-cp35m-linux_x86_64.whl
  

  (Adjust the file name, if necessary.)

That should be it!