jacksanford1/gist:d96367c18c4e467856f7c6c5274397da

## gistfile1.txt
Potential judging setup

Note: This may eventually require a 20 USDC deposit from each competitor for each contest (so people have something to lose). I think we’re moving in this direction eventually anyways.

Contest ends
3-day “public” judging period begins
Judging repo with all issues is open-sourced
For each contest, 5% of contest cost will become a judging pot (For $75k contest this looks like: $50k contest pot, $20k senior pay, $3.5k judging pot, $1.5k to Sherlock)
Anyone can comment on any issue and “judge it.” Example format:

“Severity: High
Duplicate: #12”

Severity params: High, Medium, non-issue
Duplicate params: no, #X (X represents the “lowest issue number” that’s a duplicate of this issue)

First comment (per issue) gets the credit. If the first issue is incorrect, the second, third, etc. comments are checked.

Scoring works like this:

Correct High: 5x weight in the judging pot; Incorrect High (false positive or false negative): -10x weight PENALTY
Correct Medium: 3x; Incorrect Medium (false positive or false negative): -6x
Correct non-issue: 1x; Incorrect non-issue (false positive or false negative): -2x
Correct duplicate: 1x; Incorrect duplicate (false positive or false negative): -2x
Duplicate is only correct if it references the lowest issue # that is a duplicate of the issue

Public judging ends after 3 days.
Sherlock judge comes in to check each issue. Comments on each issue with format:

“Severity: High
Duplicate: #12
Best duplicate: Yes”

The earliest comment that matches the judge’s comment is rewarded, any earlier comments that do not match the judge’s are penalized.
A script will 1) create issue labels to match the judge’s comment 2) close non-“best” duplicate issues 3) Make a new comment for each “duplicate” that re-classifies it as a duplicate of the best issue (i.e. Duplicate of #16)
Then the protocol team can review the labeled issues that are still open.

Later in the process, another script will award/deduct points from the public judging issues based on whether the issue’s final state matches a public judge’s comment.
	Potential judging setup

	Note: This may eventually require a 20 USDC deposit from each competitor for each contest (so people have something to lose). I think we’re moving in this direction eventually anyways.

	Contest ends
	3-day “public” judging period begins
	Judging repo with all issues is open-sourced
	For each contest, 5% of contest cost will become a judging pot (For $75k contest this looks like: $50k contest pot, $20k senior pay, $3.5k judging pot, $1.5k to Sherlock)
	Anyone can comment on any issue and “judge it.” Example format:

	“Severity: High
	Duplicate: #12”

	Severity params: High, Medium, non-issue
	Duplicate params: no, #X (X represents the “lowest issue number” that’s a duplicate of this issue)

	First comment (per issue) gets the credit. If the first issue is incorrect, the second, third, etc. comments are checked.

	Scoring works like this:

	Correct High: 5x weight in the judging pot; Incorrect High (false positive or false negative): -10x weight PENALTY
	Correct Medium: 3x; Incorrect Medium (false positive or false negative): -6x
	Correct non-issue: 1x; Incorrect non-issue (false positive or false negative): -2x
	Correct duplicate: 1x; Incorrect duplicate (false positive or false negative): -2x
	Duplicate is only correct if it references the lowest issue # that is a duplicate of the issue

	Public judging ends after 3 days.
	Sherlock judge comes in to check each issue. Comments on each issue with format:

	“Severity: High
	Duplicate: #12
	Best duplicate: Yes”

	The earliest comment that matches the judge’s comment is rewarded, any earlier comments that do not match the judge’s are penalized.
	A script will 1) create issue labels to match the judge’s comment 2) close non-“best” duplicate issues 3) Make a new comment for each “duplicate” that re-classifies it as a duplicate of the best issue (i.e. Duplicate of #16)
	Then the protocol team can review the labeled issues that are still open.

	Later in the process, another script will award/deduct points from the public judging issues based on whether the issue’s final state matches a public judge’s comment.