Build Tensorflow from source, for better performance on Ubuntu.

build-tensorflow-from-source.md

Building Tensorflow from source on Ubuntu 16.04LTS for maximum performance:

TensorFlow is now distributed under an Apache v2 open source license on GitHub.

On Ubuntu 16.04LTS+:

Step 1. Install NVIDIA CUDA:

To use TensorFlow with NVIDIA GPUs, the first step is to install the CUDA Toolkit as shown:

wget -c -v -nc https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/cuda-repo-ubuntu1604_9.2.88-1_amd64.deb

sudo dpkg -i cuda-repo-ubuntu1604_9.2.88-1_amd64.deb

sudo apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/7fa2af80.pub

sudo apt-get update

sudo apt-get install cuda

Keep checking the NVIDIA CUDA webpage for new releases as applicable. This article is accurate as at the time of writing.

Ensure that you have the latest driver:

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update && sudo apt-get -y upgrade

On Ubuntu 18.04LTS, this should be enough for the device driver:

sudo apt-get install nvidia-kernel-source-396 nvidia-driver-396

Failure to do this will result in a broken driver installation.

When done, create a library configuration file for cupti:

/etc/ld.so.conf.d/cupti.conf

With the content:

/usr/local/cuda/extras/CUPTI/lib64 

Confirm that the library configuration file for CUDA libraries also exists with the correct settings:

/etc/ld.so.conf.d/cuda.conf

The content should be:

/usr/local/cuda/lib64

When done, load the new configuration:

sudo ldconfig -vvvv

Useful environment variables for CUDA:

Edit the /etc/environment file and append the following:

CUDA_HOME=/usr/local/cuda

Now, append the PATH variable with the following:

/usr/local/cuda/bin:$HOME/bin

When done, remember to source the file:

source /etc/environment

You can also install CUDA manually. However, take care not to install its' bundled driver.

Step 2. Install NVIDIA cuDNN:

Once the CUDA Toolkit is installed, download the latest cuDNNN Library for Linux, based on the CUDA version you're using. In this case, we're on CUDA 9.1, so we will refer to the version name below (note that you will need to register for the Accelerated Computing Developer Program).

Once downloaded, uncompress the files and copy them into the CUDA Toolkit directory (assumed here to be in /usr/local/cuda/ for Ubuntu 16.04LTS):

$ sudo tar -xvf cudnn-9.1-* -C /usr/local

Step 3. Install and upgrade PIP:

TensorFlow itself can be installed using the pip package manager. First, make sure that your system has pip installed and updated:

$ sudo apt-get install python-pip python-dev
$ pip install --upgrade pip

Step 4. Install Bazel:

To build TensorFlow from source, the Bazel build system (and the latest available openjdk) must first be installed as follows.

$ sudo apt-get install software-properties-common swig
$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer
$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://storage.googleapis.com/bazel-apt/doc/apt-key.pub.gpg | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install bazel

Step 5. Install TensorFlow

To obtain the best performance with TensorFlow we recommend building it from source.

First, clone the TensorFlow source code repository:

$ git clone https://github.com/tensorflow/tensorflow
$ cd tensorflow

The last step is no longer needed:

$ git reset --hard a23f5d7 

Then run the configure script as follows:

$ ./configure

Output:

Please specify the location of python. [Default is /usr/bin/python]: [enter]
Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] n
No Google Cloud Platform support will be enabled for TensorFlow
Do you wish to build TensorFlow with GPU support? [y/N] y
GPU support will be enabled for TensorFlow
Please specify which gcc nvcc should use as the host compiler. [Default is /usr/bin/gcc]: [enter]
Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]: 8.0
Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: [enter]
Please specify the Cudnn version you want to use. [Leave empty to use system default]: 5
Please specify the location where cuDNN 5 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: [enter]
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size.
[Default is: "3.5,5.2"]: 5.2,6.1 [see https://developer.nvidia.com/cuda-gpus]
Setting up Cuda include
Setting up Cuda lib64
Setting up Cuda bin
Setting up Cuda nvvm
Setting up CUPTI include
Setting up CUPTI lib64
Configuration finished

Then call bazel to build the TensorFlow pip package:

bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda //tensorflow/tools/pip_package:build_pip_package

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

This will build the package with optimizations for FMA, AVX and SSE.

To build the C library as a tarball (which you can install as needed) with the optimizations above:

bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=cuda //tensorflow/tools/lib_package:libtensorflow

Which should produce an archive in:

bazel-bin/tensorflow/tensorflow/tools/lib_package/libtensorflow.tar.gz

A stock build would be as such:

bazel build //tensorflow/tools/pip_package:build_pip_package

bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg

You can use the stock build as shown above if you had passed the configuration flags (for optimization) directly to the configure script above. Use this string:

--copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2

Which will replace --march=native (the default).

If you're on Skylake to Coffee lake, this is what you need.

And finally install the TensorFlow pip package

For Python 2.7:

$ sudo pip install --upgrade /tmp/tensorflow_pkg/tensorflow-*.whl

Python 3.4:

$ sudo pip install --upgrade /tmp/tensorflow_pkg/tensorflow-*.whl

Step 5. Upgrade protobuf:

Upgrade to the latest version of the protobuf package:

For Python 2.7:

$ sudo pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/protobuf-3.0.0b2.post2-cp27-none-linux_x86_64.whl

For Python 3.4:

$ sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/protobuf-3.0.0b2.post2-cp34-none-linux_x86_64.whl

Step 6. Test your installation:

To test the installation, open an interactive Python shell and import the TensorFlow module:

   $ cd
   $ python


>>> import tensorflow as tf
tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally

With the TensorFlow module imported, the next step to test the installation is to create a TensorFlow Session, which will initialize the available computing devices and provide a means of executing computation graphs:

>>> sess = tf.Session()

This command will print out some information on the detected hardware configuration. For example, the output on a system containing a Tesla M40 GPU is:

>>> sess = tf.Session()
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties:
name: Tesla M40
major: 5 minor: 2 memoryClockRate (GHz) 1.112
pciBusID 0000:04:00.0
Total memory: 11.25GiB
Free memory: 11.09GiB

To manually control which devices are visible to TensorFlow, set the CUDA_VISIBLE_DEVICES environment variable when launching Python. For example, to force the use of only GPU 0:

$ CUDA_VISIBLE_DEVICES=0 python

You should now be able to run a Hello World application:

    >>> hello_world = tf.constant("Hello, TensorFlow!")
    >>> print sess.run(hello_world)
    Hello, TensorFlow!
    >>> print sess.run(tf.constant(123)*tf.constant(456))
    56088
    

Tips:

To achieve similar results without building the packages, you can deploy nvidia-docker and install tensorflow from NVIDIA's NGC registry.

Use this to deploy nvidia-docker on Ubuntu: https://gist.github.com/Brainiarc7/a8ab5f89494d053003454efc3be2d2ef

Use the NGC to deploy the preconfigured containers. Optimized builds for Tensorflow, Caffe, Torch, etc are also available: https://www.nvidia.com/en-us/gpu-cloud/deep-learning-containers/

Also see the NGC panel: https://ngc.nvidia.com/registry

Hello

thanks for this guide, this worked for me :)
Can I delete the source folder and package folder after pip installing?
@third-meow All you need is the .whl file to install it again in the future so just make sure not to lose that if you decide to delete the source folder or you'll have to repeat the build.

Hi,

Thanks a lot for the guide!
Would you happen to have benchmark to support the claim (I think it is true, but how much do you gain between installing from Pip, conda and build)?

Benchmark between pip and conda was explain lately (https://towardsdatascience.com/stop-installing-tensorflow-using-pip-for-performance-sake-5854f9d9eb0c) but what about the performance gain from building from source?

'''
Use this string:

--copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2
Which will replace --march=native (the default).
'''

Can you please clarify the above statement ?
IIUC , if i am building on a machine that deosn't support avx with flags --copt=-mavx and --march=native , will that throw error or it wont use avx instructions since it is not supported by native arch ?
is it neccessary to provide both --copt and --march both ? or does tensorflow automatically chooses all possible best instructions for a given architecture ?
what if i am building for an architecture that supports avx and avx2 , but i dont provide --copt=-mavx --copt=-mavx2 , but i do provide --march=native ? is this an optimized built or not ?

on a similar note , i do see option for march=native , when i do ./configure command , then i see march=native in some .bazelrc file and then the third option is providing -march=native during bazel build command ? Can you please explain the relation and which one takes precedence ?
i see ./configure probably generates some bazelrc file that is read by the bazel build as an input , but not entirely sure .

@motiteux,

Conda builds are indeed better optimized for performance (on Intel platforms), since they build with the Intel MKL libraries.
Building from source overcomes that liimitation for scenarios where MKL is either undesired or of no practical use (non Intel processor scenarios, for one).

@patelprateek,

On virtually all modern processors (post Skylake+, for Intel), these instructions are provided.
For Pre-Skylake (up to Haswell and Broadwell), you can omit mavx2.

For the native target, confirm what optimizations are availed to you:

gcc -march=native -Q --help=target

To see target specific optimizations:

gcc -march=native -Q --help=target | grep enable

Should march native fail for you, try adding --cxxopt=-march=native instead and retest.

After I built this my Tensorflow still complains that I don't use AVX512F. Is there a way to build with AVX512F support?

@alex-petrenko,

What processor are you using?
And on what Linux-based system? What version of GCC is available to you?
Show me the output of:

gcc -march=native -Q --help=target

With that information present, assuming that your GCC version is up to date (and implements AVX512F support), pass the appropriate flags to bazel at the configuration page:

bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --copt=mavx512f --config=cuda //tensorflow/tools/pip_package:build_pip_package

For information on tuning options available for x86, see this: http://gcc.gnu.org/onlinedocs/gcc/x86-Options.html

@Brainiarc7,

Do you have any numbers to show just how much of a speed up you achieve by doing your own build?

@CovertKoala,

I can provide these stats if needed.
The biggest jump observed so far was on a Xeon Platinum 8160, where these AVX-512F enablements really paid off.

@Brainiarc7,

I'm a total noob to ML and tensorflow, the question is more out of my own curiousity (no need to go out of your way to get them if you don't have them). As I play around with different NN architectures, I'm wishing things were a bit faster as I make my tweaks.

I've got an NVIDIA GTX 1080 and Intel 8th gen i7. While it definitely is quick, it's no Xeon Platinum.

Thanks!