Skip to content

Instantly share code, notes, and snippets.

@dtsmith2001

dtsmith2001/tf2-raspi-buster.md

Last active December 15, 2019 02:14
Show Gist options
  • Save dtsmith2001/81d91e5f2cb8deecd25e40dd5b0e84c8 to your computer and use it in GitHub Desktop.
Save dtsmith2001/81d91e5f2cb8deecd25e40dd5b0e84c8 to your computer and use it in GitHub Desktop.
Download ZIP
Building Tensorflow 2.0.0 on the Raspberry PI with Buster
Raw
tf2-raspi-buster.md

Building Tensorflow 2.0.0 on the Raspberry PI with Buster

These instructions were inspired by

Updated information is included based on Tensorflow 2.0.0 and Bazel 0.26.1. As far as I know, the latest acceptable version of Bazel is 0.26.1.

These instructions are written so they may be automated in a Docker container or in a shell script. I have not cross-compiled Bazel, and I don't know if it's possible.

The Tensorflow build may be cross-compiled as well. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/pi/build_raspberry_pi.sh. This script references Trusty as the host OS, so it's not clear it is appropriate for builds on Bionic. Additionally, the script builds its own OpenBLAS. I've already got that installed for OpenCV.

Following these instructions will probably take you two days.

Notes

These builds are cpu-intensive. Recommendations are

  • Do this on a PI that is not overclocked.
  • If your PI is overclocked, use active cooling and a heatsink.
  • Take the top of the case off your PI and use a fan on it.
  • Limit the Bazel jobs to 1 or 2.
  • If you can, cross-compile using an AWS instance (or your favorite cloud provider).

For AWS users

  • Do your preliminary work on a free tier instance. You do not want to pay to run apt and pip.
  • When you are finished with setting up the source tree, stop the instance.
  • Click on the instance, then Actions | Image | Create Image.
  • Give your image a name and include the date in either the image name or description.
  • Wait for the image to build (see Images | AMI).
  • Click on your instance, then Actions | Instance | Instance Settings | Change Instance Type (note that you cannot change the instance type for a spot instance).
  • Change the instance type. First, research the type of instance you need, and the spot price available. You do not need a lot of bandwidth, so don't pay for it.
  • Depending on your situation, use a spot instance but note you can't shut down a spot instance, you have to terminate it.

Prerequisites

Python 2.7 is installed by default here, and that is what the Tensorflow build needs.

sudo apt-get install pkg-config zip g++ zlib1g-dev unzip default-jdk autoconf automake libtool python

Day 1 - Build Bazel

Download Bazel 0.26.1

We are working with Bazel version 0.26.1, which works with Tensorflow 2.0.0. So first download bazel-0.26.1-dist.zip.

mkdir bazel-0.26.1 && cd bazel-0.26.1
wget https://github.com/bazelbuild/bazel/releases/download/0.26.1/bazel-0.26.1-dist.zip
unzip bazel-0.26.1-dist.zip

Adjust Swap Space

Now we have to add sufficient swap space. You can use a flash drive for swap instead of the SD card. See Step 2 for further information.

sudo dphys-swapfile swapoff
sudo sed -i -e 's/CONF_SWAPSIZE=100/CONF_SWAPSIZE=1024/' -e 's/\#CONF_SWAPFILE/\/path\/to\/swap/' /etc/dphys-swapfile
sudo dphys-swapfile setup
sudo dphys-swapfile swapon

You do not have to reboot your PI.

The Bazel Build

The build takes a long time. Put it in the background so you can log out of your PI. If you prefer doing this build from the console, nohup is not needed.

export BAZEL_JAVAC_OPTS="-J-Xmx1g -J-Xms200m"
nohup env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh > ../bazel_build.log 2>&1 &

Day 2 - Build Tensorflow 2.0.0

Editing tensorflow/core/platform/platform.h is no longer necessary.

git clone --recurse-submodules https://github.com/tensorflow/tensorflow.git
cd tensorflow
git checkout v2.0.0
grep -Rl 'lib64' | xargs sed -i 's/lib64/lib/g'
export TF_NEED_GCP=${TF_NEED_GCP:-0}
export TF_NEED_HDFS=${TF_NEED_HDFS:-0}
export TF_NEED_OPENCL=${TF_NEED_OPENCL:-0}
export TF_NEED_OPENCL_SYCL=${TF_NEED_OPENCL_SYCL:-0}
export TF_NEED_TENSORRT=${TF_NEED_TENSORRT:-0}
export TF_NEED_NGRAPH=${TF_NEED_NGRAPH:-0}
export TF_NEED_JEMALLOC=${TF_NEED_JEMALLOC:-1}
export TF_NEED_VERBS=${TF_NEED_VERBS:-0}
export TF_NEED_MKL=${TF_NEED_MKL:-1}
export TF_DOWNLOAD_MKL=${TF_DOWNLOAD_MKL:-1}
export TF_NEED_MPI=${TF_NEED_MPI:-0}
export TF_ENABLE_XLA=${TF_ENABLE_XLA:-1}
export TF_NEED_AWS=${TF_NEED_AWS:-0}
export TF_NEED_GDR=${TF_NEED_GDR:-0}
export TF_CUDA_CLANG=${TF_CUDA_CLANG:-0}
export TF_SET_ANDROID_WORKSPACE=${TF_SET_ANDROID_WORKSPACE:-0}
export TF_NEED_KAFKA=${TF_NEED_KAFKA:-0}
export TF_DOWNLOAD_CLANG=${TF_DOWNLOAD_CLANG:-0}
export TF_NEED_IGNITE=${TF_NEED_IGNITE:-0}
export TF_NEED_ROCM=${TF_NEED_ROCM:-0}
export NCCL_INSTALL_PATH=${NCCL_INSTALL_PATH:-/usr}
export PYTHON_BIN_PATH=${PYTHON_BIN_PATH:-"$(which python3)"}
export PYTHON_LIB_PATH="$($PYTHON_BIN_PATH -c 'import site; print(site.getsitepackages()[0])')"
export PI_COPTS="--copt=-march=armv7-a --copt=-mfpu=neon-vfpv4
  --copt=-std=gnu11 --copt=-DS_IREAD=S_IRUSR --copt=-DS_IWRITE=S_IWUSR
  --copt=-O3 --copt=-fno-tree-pre --copt=-fpermissive
  --copt=-DRASPBERRY_PI=1
  --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_1
  --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_2
  --copt=-U__GCC_HAVE_SYNC_COMPARE_AND_SWAP_8
  --define=raspberry_pi_with_neon=true"
./configure
bazel build --jobs=1 --local_ram_resources=HOST_RAM*.25 --local_cpu_resources=HOST_CPUS*.5 --javacopt="-Xms256m -Xmx750m" --jvmopt="-Xms256m -Xmx750m" -c opt ${PI_COPTS} --config=v2 --config=noaws --config=noignite --config=nokafka --copt=-funsafe-math-optimizations --copt=-ftree-vectorize --copt=-fomit-frame-pointer --define tensorflow_mkldnn_contraction_kernel=0 --verbose_failures //tensorflow:libtensorflow.so

Day 3 - Manual Installation of Tensorflow

This information is from https://github.com/node-tensorflow/node-tensorflow/blob/master/tools/install.sh, but it may change in the future. According to the man page, -a is equivalent to -dR, and -d is equivalent to --no-dereference --preserve=links. -R is recursive.

# Copy .h & .cc files
cp -a tensorflow/bazel-genfiles/. 				$srcDir
cp -a tensorflow/tensorflow/cc 					$srcDir/tensorflow
cp -a tensorflow/tensorflow/core 				$srcDir/tensorflow
cp -a tensorflow/google/protobuf/src/google		$srcDir
cp -a tensorflow/third_party					$srcDir

# Copy Libraries (.o)
mkdir -p $libDir/google
mkdir -p $libDir/external
mkdir -p $libDir/tensorflow

cp -a tensorflow/bazel-bin/google/.			$libDir/google
cp -a tensorflow/bazel-bin/external/.		$libDir/external
cp -a tensorflow/bazel-bin/tensorflow/.		$libDir/tensorflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment