17
Nov
Benchmarks can only grade known science
OpenAI's BioBasicsBench was one of four science benchmarks in the newsletter this week, and like the others it can only grade answers someone already knows. That makes benchmarks surrogate endpoints for discovery, and more realistic tasks won't change it.
3 min read