Skip to content

Instantly share code, notes, and snippets.

@jart jart/build-tf-nightly.sh
Last active Jan 13, 2018

Embed
What would you like to do?
TensorFlow Nightly Ubuntu 14 Workaround Ops
#!/bin/bash
# TensorFlow Nightly Python 2.7 Slow CPU Package Builder on Ubuntu 14
#
# This script is a temporary hackish workaround until the TensorFlow team can
# setup a heterogeneous tooling environment that builds Python<=3.4 CPU pip
# packages using Ubuntu 14, due to challenging constraints imposed by vendors.
#
# It uploads poor quality builds of TensorFlow to GCS for testing environments:
#
# pip install --no-cache-dir -I https://tensorboard-builds.storage.googleapis.com/tf_nightly_cpu_slow_ubuntu14-123-cp27-cp27mu-linux_x86_64.whl
#
# See https://github.com/tensorflow/tensorflow/issues/15777
set -ex
WORKDIR="/home/jart/build-tf-nightly"
rm -rf "${WORKDIR}"
mkdir -p "${WORKDIR}"
cd "${WORKDIR}"
# TODO(jart): Remove echo python: cl/181706732
# TODO(jart): Remove gemmlowp flag: https://github.com/google/gemmlowp/issues/117
sudo docker run --entrypoint=/bin/bash --rm -i -v "${WORKDIR}:/pkg" ubuntu:14.04 <<'EOF'
set -ex
cat >/etc/apt/sources.list <<OMG
deb mirror://mirrors.ubuntu.com/mirrors.txt trusty main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt trusty-updates main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt trusty-backports main restricted universe multiverse
deb mirror://mirrors.ubuntu.com/mirrors.txt trusty-security main restricted universe multiverse
OMG
apt-get update
apt-get install -y --no-install-recommends build-essential python python-dev pkg-config zip g++ zlib1g-dev unzip wget git-core python-numpy python-pip
pip install -U pip wheel
wget -q https://mirror.bazel.build/github.com/bazelbuild/bazel/releases/download/0.9.0/bazel-0.9.0-installer-linux-x86_64.sh
chmod +x bazel-0.9.0-installer-linux-x86_64.sh
./bazel-0.9.0-installer-linux-x86_64.sh
git clone https://github.com/tensorflow/tensorflow
cd tensorflow
echo 'export PYTHON_BIN_PATH="/usr/bin/python"' >tools/python_bin_path.sh
sed -i -e "s@project_name = 'tensorflow'@project_name = 'tf_nightly_cpu_slow_ubuntu14'@" tensorflow/tools/pip_package/setup.py
sed -i -e "s@_VERSION = '.*'@_VERSION = '123'@" tensorflow/tools/pip_package/setup.py
bazel --batch build --copt='-DGEMMLOWP_ALLOW_SLOW_SCALAR_FALLBACK' //tensorflow/tools/pip_package:build_pip_package
bazel-bin/tensorflow/tools/pip_package/build_pip_package /pkg
EOF
gsutil cp -a public-read "${WORKDIR}/tf_nightly_cpu_slow_ubuntu14-123-cp27-cp27mu-linux_x86_64.whl" gs://tensorboard-builds/
#!/bin/bash
# GCE Instance Setup Script
set -ex
packages=(
bash-completion
build-essential
emacs25-nox
git
htop
openjdk-8-jdk
python
python-dev
python-pip
python-virtualenv
python3-pip
python3-setuptools
screen
unattended-upgrades
zip
)
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get install -y "${packages[@]}"
crontab - <<'EOF'
# m h dom mon dow command
10 2 * * * /home/jart/build-tb-nightly.sh >/home/jart/build-tf-nightly.log 2>&1
EOF
wget https://gist.githubusercontent.com/jart/2ac6cbd4558fecece9f094e436632bdf/raw/487e48e315de627f32121af011bdd8c148a596b7/.screenrc
wget https://github.com/bazelbuild/bazel/releases/download/0.7.0/bazel-0.7.0-installer-linux-x86_64.sh
bash bazel-0.7.0-installer-linux-x86_64.sh --user
# https://docs.docker.com/engine/installation/linux/docker-ce/debian/
sudo apt-get install -y --no-install-recommends apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88 |& grep '9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88'
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install -y --no-install-recommends docker-ce
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.