The goal of this document is to describe the desired user experience for the next generation of "sig" and it's predecessor "git-signatures"
These were useful prototypes but significant improvement is needed before widespread use.
Sig already uses PGP signatures, but with no metadata to signify the what the review means.
Was it a cursory look? A deep dive? How familiar with the subject matter is the reviewer?
What if they didn't even look at the code but at least compiled it and verified it is reproducible. This should still be signed too, but it is different type of review than reading the code.
The introduction of metadata tags allowing for a normalized way to communicate review quality have emerged in similar tool and spec attempts.
Notably:
Of note, Lance has also had exposure to a number of proprietary tools developed in house for various companies.
A wide range of metadata tags may be useful for specific environments but the ones most useful in practice seem to be:
- thoroughness
- understanding
- rating
- reproducibility
Once one chooses to include metadata, then a simple detached signature no longer suffices. The tools and specs above attempt to solve this by having an opinionated format hard-coding various metadata tags and then allowing the inclusion of a chosen signature with that report. Crev, git-wotr, and basically all signatures on popular linux packages like debian, fedora, arch, etc, all take this approach.
The trouble with this approach of many one-off specs is it will forever require complex custom tooling, when the ability to add custom metadata tags directly into OpenPGP signatures already exists and can solve for these problems.
The only limiting factor is that doing any custom development to take advantage of OpenPGP features like this historically required dealing with GnuPG which is notoriously unfriendly to modern development sensibilities.
Thankfully, this landscape has changed with the sequoia-pgp team making a modern well tested and library-first implementation of OpenPGP in rust, allowing for us to have a clear path to use the OpenPGP standard to the fullest.
There is now a clear path to implement a simple scheme for detached OpenPGP signatures that contain any review metadata desired.
While the value of the OpenPGP standard and its portability and compatibility with hardware signing devices can not be under-stated, the UX of going from "I have a new Yubikey in my hand and want to review code" to "there is a published signature" is so wide that most give up instantly and seemingly blame the OpenPGP standard for ux failings of GnuPG.
Both of these problems are not an issue with OpenPGP, but a problem with the UX of the tools.
Historically this was not easy to solve because GnuPG didn't have a well developed library to easily embed workflows for an easy and portable user experience in cases like these.
The existence of a modern library like sequoia-pgp create a path for a tool like "sig" to fully streamline the first user experience of key creation and publishing so users can use PGP correctly without actually having to care about it when they are trying to get code review done.
Sig should be a one-stop to set users up for success with PGP so they can get their review done, and also be setup to take advantage of all the other wins of PGP with other tools when they are ready.
Here is the outcome we want to target for an streamlined hardware-signed code review process whose resulting signing keys and artifacts can be easily published and discovered so every review can help secure the entire open source supply chain.
- cd to directory containing any open source project you wish to review
- Run "sig review"
- All code canges since the last signed review you made and HEAD will be displayed as a diff in chosen git difftool
- (template launches to submit review)
- User fills in thoroughness, understanding, and rating boxes
- User saves.
- If needed: "Please insert a PGP compatible smartcard such as a yubikey 5"
- If needed: "This smartcard has no keys. Would you like to generate a new set? [Y/n]"
- If needed: "Keys generated for UID foo@bar.com. Publish to keyservers? [Y/n]"
- If needed: "Keys published to x, y, x servers. For maximum visibility consider publishing to popular locations like Github, Gitlab, Keyoxide, or to your own domain via WKD."
- "Signing! Please confirm on your "Yubico 5A" USB device."
- "Do you want to save this signature as a Git Note on this Git repository?"
- "Would you like to publish this signature to a public signing database so others can benefit from your review?"
Assuming simple detached signatures exist, and are optionally published to git notes or a public database, they should now be readily discoverable by "sig verify".
Every organization will have different preferences on verification.
For example a strict org worried about supply chain attacks may want the following before production use:
- 1+ review by a release engineer with keys from set a with minimum "reproduciblity" of 1
- 2+ reviews by a internal software engineer from set b with minimum "thoroughness" of 5 and "understanding" of 5
- 1+ external review done by trusted entities from set c with a minimum "thoroughness" of 2 and "understanding" of 2
"sig verify" could output any signatures found, but it would be more useful if you could set a standardized policy file in a root repo at an organization to ensure your root repo (and all dependencies) meet the desired policy.
This could prevent untrusted code from ever touching production putting a full stop to many classes of supply chain attack.
I proposed a simple format that could meet these needs as json or yaml:
[
{
"name": "release-engineers",
"min": 1,
"members": [
"05F597E88FAD0763449F8D1F573FD879821C2735",
"097722C2A2EF2E7AFB2D0C345902E7D5FB4E1ECD"
],
"metadata": {
"thoroughness": {
"min": 2
},
"understanding": {
"min": 4
},
"rating": {
"present": true
}
}
},
{
"name": "engineers",
"min": 3,
"members": [
"1351878A47640D0812452E5057546E564D259DBB",
"FE9BE6F2F92C4A3B536D326FCC3160C3C54E50BB",
"4E27CF523A3880CD4FD3B4532D147A3EE202DBBE"
],
"metadata": {
"thoroughness": {
"min": 1
},
"understanding": {
"min": 2
},
"rating": {
"present": true
}
}
}
]
Wiktor of the sequioa-pgp team graciously implemented a proof of concept of this here:
https://gitlab.com/wiktor/lance-verifier#lance-verifier
Note: Lance did not name it.
- Do we integrate the work from wiktor directly into the sequoia-pgp library and/or the sq command line app?
- Arbitrary policy based m-of-n verification seems generally useful far beyond just code review use cases
- Do we keep sig as a simple bash wrapper for tools like gnupg or sq, or rewrite in rust importing sequoia-pgp
- The answer to #1 will impact #2 here.
- Lance is not yet very good at rust so this may slow progress, but not opposed to getting more experience here.
- We need to spec and develop a public database to publish and discover open source signatures sig makes
- Should also be able to harvest and index existing signing formats best effort from linux distros, Crev, etc.
- "sig verify" could be able to pull down and use a crev or debian signature to satisfy one external signer, etc.
- Lance would default to deveoping this in PostgREST for a highly testable SQL only implementation
- Anyone could mirrored to a blockchain or ipfs for long term durability as desired
Currently we have none, so this will likely not go very quickly.
We know of several independent security engineers who would love to work on this at least part time if we had some funding to offset other opportunities.
If you or your organization sees supply chain attacks as a real threat and wants to accelerate efforts for an open toolchain to make signed and distributed code review of open source dependencies easier for everyone, contact lance@lrvick.net.
- Lance Vick - Distrust, LLC
- Wiktor - P==P, Sequoia PGP