A friendly wager between Steven Haddox and myself.

wager.textile

Wager: loser donates $100 to the winner’s named charity.

Setup:

• A reference list of the top 10,000 Alexa domains will be used (“List”).
• Ten domains are selected at random from the List.
• Ten government websites (any domain containing “.gov.” or ending in “.gov”) are selected at random from the List.
• The twenty domains are paired, one from each group, into ten pairs.
• Random numbers used for the selection will be generated by random.org.

Judging:

• We each pick a trusted expert, denoted A and B.
• A and B together agree on a third expert, C.

Decisions:

• For each pair, A and B decide independently which of the two sites is likely to have a better user experience, using only a screenshot of the domain.
• If A and B disagree, they agree to abide by C’s decision, which will be final.

Winners:

• John wins if the experts pick a greater number of government websites than non-government websites.
• Steven wins if the experts pick a greater number of non-government websites than government websites.
• If there’s a tie, Steven and John split the difference and pay $50 each to the other’s charity.

I like the heart of this, but my concerns are illustrated by the following: Of the top 500 domains in Alexa's index, only one .gov exists. That's the nih.gov website (#337 iirc). That means that you're picking the single most visited government site and pitting it against the top regular sites, and a bunch of not-so-good high traffic sites (see chaturbate). I'm not sure how to mitigate the ratio issues here where the cream of the .gov domains is pitted against a wide array of all other TLDs?

Also, I have qualms with judging solely off a screenshot. I feel semantic HTML and UI/UX best practices in CSS, etc. are important and can't be judged solely upon appearance.

Those are the only two issues I can think of off-hand...

It's true that government websites are generally less popular than non-government websites, and there are also fewer of them, but I'm not sure what the objection is. UK domains are less popular than US domains. Does that make the two incomparable?

I agree that screenshot is less than ideal. But it has to be something where we both know for sure that the judges are looking at the same thing. If it's a live link, then what if there's differences between browsers, content based on time of day, site happens to be down at that moment, etc.?

@fj: I'll yield on randomly choosing websites based upon *.gov vs other TLDs if you'll yield on the screenshot for a live link. I want the judges to be able to analyze the underlying code of the site. Obviously if a site is down then it shouldn't be considered for judgment at that moment in time (technically this argument could apply to a screenshot while the site is down too..) I'd like to hope the judges we both choose are wise enough to be able to choose appropriate browsers for testing the sites in.