Skip to content

Instantly share code, notes, and snippets.

@AP97
AP97 / spack.md
Last active February 26, 2017 18:21
Submission of SpackProject

TensorFlow is a popular machine learning library, but installing it on HPC systeems is very difficult. The Bazel build systeem makes assumptions about systems that are not always true for supercomputers (compiler locations, etc.) and it fetches files from the internet, which doesn't work on airgapped networks.

Spack is a package manager designed for supercomputers. It can build optimized versions of packages with fast compilers. We decided to port the Tensorflow build to Spack. We packaged three Google packages (farmhash, highwayhash, and gemmlowp) to Spack, and we submitted pull requests for all three after verifying that they built correctly.

We also developed a prototype package for TensorFlow, which has 45 transitive depndencies. We needed to fool the