github's rendering of org mode sucks (links, footnotes and outline layout are all kind of broken), so this gist is best cloned and viewed in emacs.
once my editing settles down this will be converted to markdown or mediawiki format for better viewing on github. apologies!
Git objects are hashed with SHA1, for which full collisions have been found, but they do commit to the length. since existing attacks are only collision, not 2nd preimage attacks, this still provides some level of security, especially if source trees are frequently updated.
Recently, merge commits also include a Tree-SHA512 value corresponding to
the the tree object’s contents. Since commits are signed, they can be verified
using a more secure hash, contingent on being able to validate and trust
signatures made by the repository maintainers[fn:ots].
The maintainers’ trusted keys can be considered an ultimate trust root for our purposes[fn:trusted-keys]. In other words the goal of verifying a release is to gain certainty that a specific binary distribution of the software corresponds to source code signed with these keys.
[fn:ots] Merge commits are also securely timestamped
[fn:trusted-keys] We assume the Bitcoin Core contribution process defines the specific versions of the code that the user intends to run, and that the user trusts the maintainers of the repository to only sign commits and tags accordingly.
The details of authenticating the keys themselves, including defining and establishing a trusted computing base are beyond the of scope for this document.
Risks can be mitigated by introducing redundancy (i.e. comparing values obtained using out of band channels or anonimity networks with others, and utilizing separate pieces of hardware from multiple vendors).
Bitcoin Core releases[fn::in the sense referred to by the release field in the
build assert files] originate from corresponding git tag, which is just the
version and option release candidate number, prefixed with a v, e.g.
v0.16.0rc1. Releases are further identified by platform, e.g.
0.16.0rc1-linux or 0.16.0-win.
Releases are built using gitian which (hopefully) makes the build products reproducible, providing some assurances that the compiled software actually corresponds to the source code.
A build results in multiple output files, whose names include the architecture
but omit release candidate info (so e.g. 0.16.0rc1-linux would be labeled
0.16.0-x86-linux-gnu, with the full name being
bitcoin-0.16.0-x86-linux-gnu.tar.gz)[fn::debug symbols are also provided as
separate outputs].
The gitian.sigs repository is essentially a database keyed by release string, subkeyed by builder handle, with assertions about the build results provided by different gitian builders - YAML files with hashes comitting to the contents of all the relevant files, and accompanying PGP signatures from the builders.
The Bitcoin release process[fn::out of date, links to https://github.com/bitcoin-core/bitcoin-detached-sigs] explains that once there’s consensus about the hash of the tarball, an official release can be made.
This implies comparing the outputs reproducible builds[fn::i.e. an “official” release consists of the combination of the different per platform releases] as asserted by the different builders, their hardware, operating system vendors, and the security of the keys with which they sign the assertions.
Once all build products are gathered, a SHA256SUMS file comitting to all of
them can be made and signed with the official binary release signing key, this
new SHA256SUMS.asc along with the build products is published on bittorrent,
https://bitcoincore.org, and https://bitcoin.org, with release candidates are
published in a separate subdirectory (e.g. test.rc1) of the intended release
(e.g. bitcoin-core-0.16.0).
In other words the release key certifies by signing the SHA256SUM file that
the released files match some qurom of gitian builders, as verified by their
signatures. This signature defines an official correspondence between a bitcoin
core release identifier, a specific source tree and a specific set of binary
distribution tarballs.
The bitcoin repository includes a script that downloads and validates
SHA256SUMS.asc assuming the release key has been verified by the user.
A user who wishes to run the specific binaries corresponding to an official
release (as defined by a git tag) may obtain these binaries either using the
gitian-build script to attempt produce the desired tarball themselves, by
downloading them (or both).
The user may trust the release key to specify a source tree separately from the trusted keys used to sign merge commits, or they may compare the signed tags with the released source tarballs.
The user may rely on the release key to define the expected hashes of the binary distribution tarballs, trusting that there were no discrepancies with the gitian builds, or they can compare it to the assertions of the various gitian builders as well their own builds.
out_manifests (gitian signer keys, git remotes)Evidence for validity or revocation status of key IDs accumulates over time, as these are generally invariant across releases. That said new keys may be introduced and old keys revoked, so freshness of ID lists must still be ensured.
A user can verify the keys using the PGP web of trust, or rely on centralized internet infrastructure to establish these initial values.
in bitcoin repo contrib/verify-commits
more private and censorship resistant ways of obtaining git repository contents
central distribution sites such as bitcoincore.org, bitcoin.org,
github.com, as well as PGP key servers (required for revocation
certificates) are reliant on varying degrees of trust in centralized
internet infrastructure
ICANN, registrars, domain owners
site and/or CA certificates could be pinned, as well as looked up on certificate transparency project
This set of documents started as an attempt to specify how an automated tool for verifying gitian signatures might work.
Upon further examination of the problem of verifying builds it appears there are several possible automated checks that could be implemented:
verify that a signed git tag matches a signed release source tarball
verify that assertion files are well formed YAML
allow whitelisting of known instances, as they are usually benign
require matching hash and signatures from some subset gitian builder, not just release key.
could optionally sign verification state
OpenTimestamps on their own can’t be used to prove repo is out to date, but does does provide necessary primitive to provide freshness checks.
If the most recent release information and key IDs were periodically timestamped, signed and published, this would allow users to confirm the freshness of their release, with similar assurances to those provided by The Update Framework[fn:tuf] (TUF), as well as potentially adding a fail closed (whitelist) mechanism to complement the existing PGP revocation certificates.
TUF/notary includes fine grained controls, including thresholds and delegation, and separates key roles into 4 layers, root (certifying), target (similar to release key), snapshot (certifying global state) and timestamp keys. Some subset of this functionality already exists.
If the repository is considered authoritative with respect to the trusted key IDs, release key ID(s), and the gitian key IDs (including certifying these keys as part of a release), and merging of a change that removes a key is considered authoritative revocation of that key, a whitelisting policy similar to TUF can be provided for users attempting to validate builds.
Furthermore, whenever the repository state changes or some specified interval elapses, the repository state (branches & tags) can be signed (either with a new timestamping key or with the code signing keys), timestamped, and published in the normal release channels.
Finally, from it seems notary could simply be used on top of current code and release signing, to provide the freshness certification mechanism[fn:tokens]. Users could use the notary client to verify the contents as well as freshness of release files and source code by querying an official server. OpenTimestamps should still be used to audit notary timestamps (and ots proofs included in notary timestamps) to further mitigate against compromise of the timestamp key (which is kept online in the server).
[fn:tuf] As far as I know using TUF or notary for bitcoin has not been discussed, but a zcash issue has some relevant discussions, e.g. that TUF does not protect against after the fact compromise of timestamping keys to alter history. By validating OTS timestamps with the released bitcoin client, timestamping key compromise can only add malicious timestamps from that point onwards.
[fn:tokens] notary supports yubikeys, but via PIV/PKCS11 functionality, not as a GPG smart card.
Since the bitcoin git repository has strong integrity and authenticity guarantees for the main branches, Git is only used as a distribution mechanism and isn’t required for securing repository contents.
Both the bitcoin and the gitian.sigs repositories could be synchronized
with additional mechanisms avoiding centralized infrastructure.
git annex provides p2p syncing over tor with SPAKE authentication (magic wormhole)
git annex also has mechanisms for fetching files specified by a strong hash using a web URL or from bittorrent, so a fork of the gitian.sigs repository with additional annexed release files could work as a resilient and secure distribution mechanism. The I2P Tahoe-LAFS grid could also be used as a git-annex special remote.
see also intro