State table for ansible-galaxy signature verification

ansible-galaxy_signature_verification.rst

	Unconfigured keyring (default)	Configured keyring
		R == -1 ("all")		R >= 1 (default)
		F == 0	F > 0	S >= R	S < R
U >= 1	Invalid option error 1	Success 2	Fail	Success	Fail
G >= 1	Warn	Success 2	Fail	Success	Fail
U == 0 and G == 0	Success	Success 3		Fail 4

1. Gives error that the keyring needs to be configured if user wants to use signature verification. I wondered if this should be a warning instead, but made it an error because it seems like this would probably be done by mistake. Signature verification can be toggled off to opt out without ambiguity.
2. If I > 0 and S == 0, there were only ignored signatures. I'm considering this a success because no non-ignored signatures could be considered "all". Should this be a warning?
3. Sucessful because no signatures could be considered "all".
4. Currently a failure because None < R, but should this just be a warning?

U

Number of user-provided signatures.

G

Number of galaxy-provided signatures.

R

Number of required successful signatures.

S

Number of signatures that are successful. This does not include signatures which fail but are ignored.

F

Number of signatures that fail. This does not include signatures which fail but are ignored.

I

Number of signatures that fail but are ignored. This is used to limit the errors a user will see and prevent certain errors from incrementing F.

Update 03-07-2022:

I added support to make signature verification strict (e.g. --required-valid-signature-count +all). The behavior on the left side of the original table (unconfigured keyring which is the default, i.e. keyring is None) is the same, so I didn't put it in this table. I put asterisks next to the things that are a change in behavior.

Assuming the keyring is not None:

	R == all	R >= 1
S == 0 and not strict	Success (F == 0)	U/G == 0	U/G >= 1
		Success **	Fail (S < R)
S == 0 and strict	Fail *	Fail
F == 0	Success	Only S is relevant to R >= 1
F > 0	Fail
S >= R	Only F is relevant to R == all	Success
S < R		Fail

* Prior to 'strict', this was a verification success. Changed to give 'all' an option to be as strict as R >= 1 and require signature verification occurs.

** Prior to 'strict', this was a verification fail because R < S. Changed to make R >= 1 more lenient, like 'all', if no signatures are found.

I edited the gist to include an updated state table for configured keyring (i.e. is not None). Wanted to highlight how this impacts your primary concerns:

• signatures always required (that can't be bypassed by just unconfiguring/removing/emptying the keyring)

For R == 1 (the default when the keyring is configured) this was the behavior prior to adding strict into the mix. Now this is not the default behavior for configured keyring (the caller will need to set the required count to +1 instead of the default (1)). I wonder if the default should be +1 now instead of 1, since having a strict default and opting-into more flexible settings seems safer.

R == +all now allows failing if no signature verification occurs (regardless of if all errors are ignored or if there were no signatures).

Removed/empty keyrings are still an error.

Unconfigured keyrings (i.e. None, not provided to ansible-galaxy) still behave as outlined in the first table.

• ensuring that all is always stricter than R>=1

I tried to find middle-ground by making +all as strict as +1 to ensure valid signature verification occurs and making 1 as lenient as all if there are no signatures. I think the latter is useful if someone expects at most 1 signature from the server to be valid but doesn't know if the collection is signed. If R == 1 and signatures are found but not successful (and are all ignored), this is still an error, which feels right but seems like a cornercase that may trip people up. Maybe for the sake of simplicity it needs to be a success regardless of whether (U == 0 and G == 0) or (G >= 1 or U >= 1).