sebbacon

Code Review: opensafely/metformin_covid

Scope: Full commit history (~120 substantive commits, Sep 2024 – May 2026)
Focus: Research methodology changes; outputs that could identify patients via moderately_sensitive namespaces
Date of review: 2026-05-24

---

Background

sebbacon

Standardized Testing & Outsourced Screening

Many commenters pointed out the temptation to revert to “controlled” environments or third-party prescreening centers to block AI-assisted cheating—but noted historical failures and trade-offs.

• jarsin suggests a federal-style testing network, “Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs.”
• recursive and benatkin debate Triplebyte’s model, with thaumasiotes noting, “No, that was Triplebyte’s marketing. Their actual product was completely different.”
• enmyj laments Karat-style outsourcing: “All of these technical interviews still suck though! I basically never code with someone watching me and find it very difficult to do in interviews.”
• AlexGizis reminds us of certification pitfalls: “Sun Microsystems pushed certifications in the 90s… Top performing employees had no time to study for the exams… And then the certification fad came mo

sebbacon

Standardized Pre-Employment Testing

Many contributors discuss the idea of an industry-wide, SAT- or ACT-style exam for tech roles—administered at test centers, with scores valid across companies—to replace or supplement current prescreening. Advocates argue this would cut costs and inefficiencies, while critics point to historical failures and the risk of privileging test-takers over top performers.

• “Standardized plus the ability for companies to do their own test after they pass the standard one. So go get prescreened at test center then use that test to apply for jobs.” — jarsin • “It’s been tried and failed: Sun Microsystems pushed certifications in the 90s… More certifications implied a worse employee.” — AlexGizis • “That’s what Triplebyte planned on. The truth is I don’t trust anyone else to run evals.” — renewiltord

sebbacon

Standardized Testing vs. Hands-On Evaluation

A significant portion of the discussion revolves around the perceived failings of standardized, "Leetcode-style" interviews compared to assessments based on candidates' real work and experience.

• Many users criticize the prevalence of memorization-based algorithmic interviews:

  "I never understood why Big Tech never setup contracts with all the SAT and ACT test centers across the country... it made sense for the recruiters to send potential candidates to a test center to do a pre assessment." (jarsin) "It’s like asking a dentist interview candidate to show you examples of fillings and crowns they did at home as a hobby. I don’t understand why there is this automatic assumption that people who program at work also do it outside of work." (ryandrake)

• Critics argue that such tests filter for those who can game the process, rather than those who'd succeed in the actual job: > "Interviewing fads are set by these large companies that have the problem of s

sebbacon

// This script runs on a Shelly switch (a Shelly 1PM Mini Gen 3) which controls my central heating boiler
// It checks an H&T gen3 sensor via the cloud
// and turns the boiler on and off to ensure we're within min/max threholds, during waking hours

let config = {
  server: "<url_from_your_shelly_cloud_account>",
  auth_key: "<key_from_your_shelly_cloud_account>",
  sensorId: "<sensor_id_from_your_shelly_cloud_account>",
  switchId: 0,
  minTemp: 18,

sebbacon

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.09kB 0.0s done
#1 DONE 0.0s

#2 [internal] load metadata for docker.io/ebmdatalab/datalab-jupyter:python3.8.1-2328e31e7391a127fe7184dcce38d581a17b1fa5
#2 DONE 0.5s

#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

sebbacon

This log output shows Chris running `python run.py` on Windows. It shows two issues. 

1. A syntax error in jupyterlab
2. A config error (presumably due to deprecated config) for jupyterlab 

```txt
(base) C:\Users\cwood\Documents\GitHub\rich_test_1>python run.py
Building docker image. This may take some time (particularly on the first run)...
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile

sebbacon

import subprocess

from PIL import Image

# This is the complex parsing. It locates every single italic character in the text, 
# and then adds a custom inline style to it
js = """document.querySelectorAll('*').forEach(element => {
    const fontStyle = window.getComputedStyle(element).fontStyle;
    if (fontStyle === 'italic') {
        element.style.backgroundColor = 'lightgrey';

sebbacon

"""Hacky script to download audio from a BBC interview and transcribe, with OpenAI tools


"""

import subprocess
from optparse import OptionParser


# Define command line options

sebbacon

Data Set	FileName	Record Count	Comments
National Waiting List	TPP_NWL_ClockStops_20231124_0819.txt	195,596	This is a cumulative load eg we only send the latest weeks data to be appended.
National Waiting List	TPP_NWL_Diagnostics_20231124_0819.txt	945,519	This is a cumulative load eg we only send the latest weeks data to be appended.
National Waiting List	TPP_NWL_OpenPathways_20231124_0819.txt	6,599,819	This is a cumulative load eg we only send the latest weeks data to be appended.
Antiviral Data	TPP_Antiviral_20231124_0823.txt	109,611	This is a full refresh of all data.
PHE Data	TPP_PHE_Tests_Positive_20231124_0827.txt	6,132,4