3 min read

Benchmarks can only grade known science

OpenAI's BioBasicsBench was one of four science benchmarks in the newsletter this week, and like the others it can only grade answers someone already knows. That makes benchmarks surrogate endpoints for discovery, and more realistic tasks won't change it.
Benchmarks can only grade known science
Photo by Navy Medicine / Unsplash

Last Monday, OpenAI released BioBasicsBench, 100 tasks that test how well AI agents handle everyday wet-lab science. A benchmark is a shared test that scores different AI systems the same way, so you can line them up and compare them. By Wednesday, the newsletter had covered three more: one on phase I trial design, one on cancer care, and one on biomedical machine learning.

Four exams in one week. Each one shares a property that is easy to miss because it is built into what a benchmark is. To score systems the same way, someone has to know the right answer before the test runs; simple, right? So a benchmark can only grade science we already know. I'd argue that makes every science benchmark a surrogate endpoint for discovery, and more realistic tasks don't change that.

Surrogate endpoints come from clinical trials. A trial can't always wait years for survival data, so it measures something faster that should track survival, like a biomarker. The surrogate is required. But it isn't the outcome you care about, and drug development has its share of drugs that moved the biomarker and missed the outcome that mattered. Once a measurement becomes the target, it stops being a good measurement.

TrialDesignBench makes the point almost on its own. It grades language models on phase I trial design by scoring their proposals against established clinical knowledge. That is a sensible way to build a test. It is also, by design, a test of whether the model agrees with what clinicians already believe. A better dosing scheme that nobody has thought of yet would likely lose points on that test.

BioBasicsBench raises the other question a trialist asks about any surrogate: who picked the endpoint? OpenAI builds AI models, and it wrote a test that AI models will be graded on. I don't think that makes the test bad. It means a company with models in the running decided what counts as everyday wet-lab competence, and every lab that compares agents on it inherits that decision.

Back in 2022, I wrote about naming things: knowing the name of a thing is not the same as knowing the thing. A benchmark scored against established knowledge checks the name. It tells you a model has learned what the field already wrote down. Discovery is the part nobody has written down yet.

On the flip side, benchmarks are getting closer to real work, and this week supplies early evidence. NPCBench tests models across the whole nasopharyngeal carcinoma care pathway, reading imaging and text together. It is framed as an apprenticeship: it grades the sequence of decisions a trainee would make, rather than isolated diagnostic questions. Keep going in that direction, the argument goes, and benchmarks will look enough like the job that the gap stops mattering.

Part of that is right. A model that can't follow a care pathway clinicians already trust shouldn't be handed one they don't. As a screen for failure, a realistic benchmark beats a quiz, and NPCBench is a better test than a list of isolated questions.

But the apprenticeship framing fails quickly in science. An apprentice is graded against the master. That works for training, because the master's decisions are the standard. It doesn't work for discovery, where the goal is to find something the master got wrong. In reality, surrogates track known practices, but fail to connect the score to an outcome nobody has measured yet.

So what should a lab do with a leaderboard? Use it the way trialists use a biomarker: to decide what is worth testing, never as the result. A high rank is a reason to try an agent at the bench. It isn't evidence the agent will find anything there.

Here's what I'd watch for in the next few months: the first paper that takes agents ranked on one of these benchmarks (BioBasicsBench is an obvious candidate) and runs what they propose as real experiments. If the leaderboard order holds at the bench, the surrogate is a good one, and I've overstated the gap. If the order scrambles, we'll know how far these scores sit from the outcome.

Four exams arrived this week, each with an answer key. I'm more interested in the first one graded by an experiment.