Run sudo ubuntu-drivers devices to find what drivers are available, and then use apt to install the driver. In my case, it is apt install nvidia-driver-390. Reboot after installation completes, and use nvidia-smi command to ensure the driver is loaded.
Use Anaconda to simplify the process of installing tensorflow and related packages. Install it with wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh and then execute it.
With Anaconda, it's a one-liner: conda install tensorflow-gpu.
If you don't wish to install it to the base conda environment, you can create a one for it with conda create tf before installoing tensorflow.
Also, if you see error like "CUDA driver version is insufficient for CUDA runtime version", the version of CUDA tools installed by Anaconda is too high for the Nvidia driver. For example, I have driver version 390, and using the compatibility table, the highest CUDA we can install is 9.1, and therefore the highest prebuilt tensorflow we can install with Anaconda is 1.9. Note that if you use Anaconda to install tensorflow-gpu, it bundles the CUDA toolkit and does not use system CUDA toolkit.
Verifying Tensorflow GPU Run python -c 'import tensorflow as tf; tf.Session()' in the Anaconda environment and you should see:
2018-11-08 09:35:02.932148: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7539 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:07:00.0, compute capability: 6.1)
Anaconda simplies the process by bundling a large denpendency tree including very big pieces such as CUDA toolkit and cuDNN. If you want to avoid Anaconda, you can do so as well.
Installing CUDA library Simply: apt install nvidia-cuda-toolkit.
Veryfing CUDA Download CUDA code sample from https://github.com/NVIDIA/cuda-samples/ and then cd into Samples/deviceQuery. Build it with SMS=72 CUDA_PATH=/usr make, and then you can invoke ./deviceQuery to make sure CUDA toolchain is successfully installed. I added SMS=72 because the system driver (390) does not support compute_75, which is listed in the Makefile.
Installing cuDNN Next is installing cuDNN. However, there is no deb package for it in the official respository. After getting the CUDA version with nvcc --version, download the appropriate cuDNN from Nvidia's website and follow Nvidia's instruction at here. If we are not compiling tensorflow, we don't need the dev package.
Installing Tensorflow Use pip to install tensorflow-gpu (with or without virtualenv). Note that the pre-built version assumes certain version of CUDA toolkit, so you might need to downgrade if the downloaded version does not work with system CUDA version.