Skip to content

Instantly share code, notes, and snippets.

@dongcarl

dongcarl/guix-README.md Secret

Last active July 11, 2019 20:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dongcarl/a9f86bb11d7b55e07b592db807430a64 to your computer and use it in GitHub Desktop.
Save dongcarl/a9f86bb11d7b55e07b592db807430a64 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
guix-README.md

Bootstrappable Bitcoin Core Builds

This directory contains the files necessary to perform bootstrappable Bitcoin Core builds.

Bootstrappability furthers our binary security guarantees by allowing us to audit and reproduce our toolchain instead of blindly trusting binary downloads.

We achieve bootstrappability by using Guix as a functional package manager.

Requirements

Conservatively, a x86_64 machine with:

  • 2 or more logical cores
  • 4GB of free disk space on the partition that /gnu/store will reside in
  • 24GB of free disk space on the partition that the bitcoin repository resides in

Note: these requirements are slightly less onerous than those of Gitian builds

Setup

If you're just testing this out, you can use the Dockerfiles for convenience. You can skip this section and it automatically speeds up your builds by using substitutes.

Otherwise, follow the Guix installation guide.

Note: For those who like to keep their filesystems clean, Guix is designed to be very standalone and will not conflict with your system's package manager/existing setup. It only touches /var/guix, /gnu, and ~/.config/guix.

Before continuing, you may want to consider adding substitute servers to speed up your build if that fits your security model. (skippable if you're using the Dockerfiles)

Once Guix is installed, deploy our patched version into your current Guix profile. The changes there are slowly being upstreamed.

guix pull --url=https://github.com/dongcarl/guix.git \
          --branch=2019-05-bitcoin-staging \
          --max-jobs=4 # change accordingly

Make sure that you are using your current profile.

export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"

Note: There is ongoing work to eliminate this guix pull step using Guix inferiors and channels.

Usage

From the top of a clean Bitcoin Core repository:

./contrib/guix/guix-build.sh

After the build finishes successfully (check the status code please), compare hashes:

find output/ -type f -print0 | sort -z | xargs -r0 sha256sum

Recognized environment variables

  • HOSTS

    Override the space-separated list of platform triples for which to perform a bootstrappable build. (defaults to "i686-linux-gnu x86_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu riscv64-linux-gnu")

    Windows and OS X platform triplet support are WIP.

  • SOURCES_PATH

    Set the depends tree download cache for sources. This is passed through to the depends tree. Setting this to the same directory across multiple builds of the depends tree can eliminate unnecessary redownloading of package sources.

  • MAX_JOBS

    Override the maximum number of jobs to run simultaneously, you might want to do so on a memory-limited machine. This may be passed to make as in make --jobs="$MAX_JOBS" or xargs as in xargs -P"$MAX_JOBS". (defaults to the value of nproc outside the container)

  • SOURCE_DATE_EPOCH

    Override the reference timestamp used for bit-for-bit reproducibility, the variable name conforms to standard. (defaults to the output of $(git log --format=%at -1))

  • V

    If non-empty, will pass V=1 to all make invocations, making make output verbose.

  • ADDITIONAL_GUIX_ENVIRONMENT_FLAGS

    Additional flags to be passed to guix environment. For a fully-bootstrapped build, set this to --bootstrap --no-substitutes. Note that a fully-bootstrapped build will take quite a long time on the first run.

Tips and Tricks

Speeding up builds with substitute servers

This whole section is automatically done in the convenience Dockerfiles

For those who are used to life in the fast (and trustful) lane, you can use substitute servers to enable binary downloads of packages.

For those who only want to use substitutes from the official Guix build farm and have authorized the build farm's signing key during Guix's installation, you don't need to do anything.

Authorize the signing keys

For the official Guix build farm at https://ci.guix.gnu.org, run as root:

guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub

For dongcarl's substitute server at https://guix.carldong.io, run as root:

wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize

Use the substitute servers

The official Guix build farm at https://ci.guix.gnu.org is automatically used unless the --no-substitutes flag is supplied.

This can be overridden for all guix invocations by passing the --substitute-urls option to your invocation of guix-daemon. This can also be overridden on a call-by-call basis by passing the same --substitute-urls option to client tools such at guix environment.

To use dongcarl's substitute server for Bitcoin Core builds after having authorized his signing key:

export ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--substitute-urls="https://guix.carldong.io https://ci.guix.gnu.org"'

FAQ

How can I trust the binary installation?

As mentioned at the bottom of this manual page:

The binary installation tarballs can be (re)produced and verified simply by running the following command in the Guix source tree:

make guix-binary.x86_64-linux.tar.xz

When will Guix be packaged in debian?

Vagrant Cascadian has been making good progress on this here. We have all the pieces needed to put up an APT repository and will likely put one up soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment