In the following, we'll assume a setting with a Git repo, master branch and pull requests (e.g. Github). We'll also assume that some tests are run on each pull request to verify correctness of a commit which is a candidate to be the next state of the master branch, of which is a direct descendant (whether this candidate is authored by a user or the result of an auto-merge between a user-authored commit and the last master branch doesn't matter).
A predicate run on the candidate commit x
. P :: Commit -> Bool
.
In principle a function Commit -> A
(e.g. running a compiler on some source code) and some predicate A -> A -> Bool
(e.g. ==
).
When the predicate returns false, we either have an error (e.g. the compiler now produces incorrect outputs for the source code), or the change is acceptable (e.g. the output changed in a way that is behaviorally equivalent). In the latter case we can override the breakage, which is known as "accepting" the test.
In practice a golden test is implemented by storing the A
from the last known accept along with each commit. We can think of the master branch as Tree (A, Commit)
.
Merges can only be accepted if the A
stored in the two branches are equal. (TODO why?)
If there is a conflict a rebase followed by an accept should be performed instead.
With a Commit -> A
(e.g. some performance measurement) and some predicate A -> A -> Bool
(e.g. >=
, meaning performance must not regress). The >=
could of course be any partial order.
Here we could similarly "accept" breakages explicitly.
TODO could we allow merges in non-equal conditions here?
This is an example of a partial order. If A
and B
are protobuf messages, A <= B
iff the keys of A is a subset of the keys in B. By checking this for all Protobuf messages in our repo, we assure that the never remove a field rule is not violated.