A short story how I did build Tensorflow 2.3 from source on Ubuntu 20.4.1 LTS with CUDA 11.0.207

tensorflow2.3_build_error.md

Building Tensorflow 2.3 from source on Ubuntu 20.4.1 LTS

GPU

ASUS ROG Strix GeForce GTX 1070 Ti 8GB

System

            .-/+oossssoo+/-.               me@ubuntu 
        `:+ssssssssssssssssss+:`           ------------------------- 
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 20.04.1 LTS x86_64 
    .ossssssssssssssssssdMMMNysssso.       Host: MS-7522 3.0 
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.4.0-47-generic 
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 5 hours, 50 mins 
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 2032 (dpkg), 6 (snap) 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: bash 5.0.17 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 1920x1080 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   DE: GNOME 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   WM: Mutter 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   WM Theme: Adwaita 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Yaru-dark [GTK2/3] 
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/    Icons: Yaru [GTK2/3] 
  +sssssssssdmydMMMMMMMMddddyssssssss+     Terminal: x-terminal-emul 
   /ssssssssssshdmNNNNmyNMMMMhssssss/      CPU: Intel Xeon X5650 (12) @ 2.668GHz 
    .ossssssssssssssssssdMMMNysssso.       GPU: NVIDIA GeForce GTX 1070 Ti 
      -+sssssssssssssssssyyyssss+-         Memory: 5393MiB / 32107MiB 
        `:+ssssssssssssssssss+:`
            .-/+oossssoo+/-.                                       

CPU:       Topology: 6-Core model: Intel Xeon X5650 bits: 64 type: MT MCP arch: Nehalem rev: 2 L2 cache: 12.0 MiB 
           flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 64159 
           Speed: 1604 MHz min/max: 1600/2668 MHz boost: enabled Core speeds (MHz): 1: 1612 2: 1607 3: 1717 4: 1606 5: 1608 
           6: 1605 7: 1604 8: 1611 9: 1607 10: 1606 11: 1606 12: 1606

Graphics:  Device-1: NVIDIA GP104 [GeForce GTX 1070 Ti] vendor: ASUSTeK driver: nvidia v: 450.51.06 bus ID: 03:00.0 
           chip ID: 10de:1b82 
           Display: x11 server: X.Org 1.20.8 driver: nvidia unloaded: fbdev,modesetting,nouveau,vesa compositor: gnome-shell 
           v: 3.36.4 resolution: 1920x1080~60Hz 
           OpenGL: renderer: N/A v: N/A direct render: N/A

Setup

gcc --version

gcc (Ubuntu 7.5.0-6ubuntu2) 7.5.0

g++ --version

g++ (Ubuntu 7.5.0-6ubuntu2) 7.5.0

bazel --version

bazel 3.1.0

cat /usr/local/cuda/version.txt

CUDA Version 11.0.207

cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 0

#define CUDNN_MAJOR 8

Build

bazel build --config=cuda --copt='-march=nehalem' //tensorflow/tools/pip_package:build_pip_package --verbose_failures

bazel build --config=opt --copt=-msse4.1 --copt=-msse4.2 --config=cuda --copt='-march=nehalem' //tensorflow/tools/pip_package:build_pip_package --verbose_failures

Success

python3 -c 'import tensorflow as tf; print(tf.config.list_physical_devices("GPU"))'

2020-09-12 06:43:55.696138: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.11.0
2020-09-12 06:43:56.733443: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2020-09-12 06:43:56.773448: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: GeForce GTX 1070 Ti computeCapability: 6.1
coreClock: 1.683GHz coreCount: 19 deviceMemorySize: 7.93GiB deviceMemoryBandwidth: 238.66GiB/s
2020-09-12 06:43:56.773494: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.11.0
2020-09-12 06:43:56.775971: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.11
2020-09-12 06:43:56.776982: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2020-09-12 06:43:56.777263: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2020-09-12 06:43:56.780368: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2020-09-12 06:43:56.781094: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.11
2020-09-12 06:43:56.781239: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudnn.so.8
2020-09-12 06:43:56.782857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]