Artificial intelligence benchmarks are meant to make progress legible. A model takes a standardized test, produces a score, and researchers, companies and customers gain a simple way to compare its performance with alternatives. But that simplicity may conceal a serious problem: a benchmark can shape the field it is supposed to measure.
Computer scientist Moritz Hardt argues that modern AI benchmarking is fundamentally broken, even though it has also helped propel machine learning forward. Hardt, a director at the Max Planck Institute for Intelligent Systems, describes benchmarks as both measuring sticks and forces that can dominate research priorities. His concern is not that standardized tests are useless. It is that their scores are too easily treated as proof of intelligence, capability or real-world reliability.
When a score becomes the goal
The central difficulty is familiar well beyond AI. Once a measure becomes a target, people and institutions learn to optimize for the measure itself. In machine learning, that can mean building systems that perform exceptionally well on a fixed dataset or leaderboard without acquiring the broader ability the benchmark was intended to capture.
Hardt has warned that machine-learning systems can improve their leaderboard standing “without doing anything intelligent.” That gap matters because benchmarks often influence research funding, public perception, product claims and technical road maps. A high score can signal a genuine advance, but it can also reflect a model’s fit to the peculiarities of a test.
Today’s proliferation of tests for large language models has made the issue more visible. New benchmarks appear regularly, claiming to assess skills ranging from specialized expertise to general intelligence. They can provide useful evidence, but a rapid cycle of benchmark creation and benchmark solving also encourages a narrow race: find the test, train toward it, surpass the previous score and move on.
A history of outsized influence
Benchmarks have repeatedly altered the trajectory of AI research. In 1987, a Defense Advanced Research Projects Agency benchmark for automatic speech-to-text translation found that leading systems did not outperform comparatively simple statistical approaches. The disappointing result contributed to an era in which interest and funding for AI research declined.
A later benchmark had the opposite effect. In 2012, ImageNet highlighted the performance of an eight-layer neural network on image classification, helping ignite the deep-learning revolution. These episodes demonstrate why benchmarks are so powerful: they do not merely summarize technical work. They can help define what counts as progress and direct the attention of an entire field.
Hardt’s argument is deliberately provocative. In his 2026 book, The Emerging Science of Machine Learning Benchmarks, he calls benchmarks “the mistake that made machine learning.” Yet he also says they contributed to legitimate progress, in part through luck. The tension is important. A flawed measurement system can still be productive, but that does not make its weaknesses safe to ignore as AI systems move into more consequential uses.
What better evaluation would require
Turning benchmarking into a more rigorous science would mean treating evaluation as an ongoing research problem rather than a final exam. Tests need to be interpreted with care: what exactly does the benchmark measure, how closely does it resemble the intended deployment, and how likely is it that models or developers have adapted specifically to it?
It would also mean resisting the temptation to compress complex capabilities into a single headline number. A model may be strong at one type of question and weak at another. It may excel on familiar formats while failing when a task changes slightly. In practical settings, researchers and decision-makers need evidence about those boundaries, not just evidence of peak performance.
The point is not to abandon standardized evaluation. Without shared tests, it would be far harder to compare methods or detect improvement. Rather, the goal is to build evaluations that are more transparent about their limits and less vulnerable to becoming targets for superficial optimization.
The stakes rise with AI adoption
As AI systems are used in more domains, benchmark claims increasingly travel beyond research labs. They can affect purchasing decisions, investment narratives and assumptions about whether a system is ready for a task. A benchmark score is therefore not only a scientific result; it can become a signal with economic and social consequences.
Hardt’s critique offers a useful corrective to the idea that an ever-rising leaderboard necessarily reflects an ever-deeper form of intelligence. Scores remain valuable, but they are pieces of evidence, not verdicts. The field’s next advance may depend as much on improving how it measures AI as on improving the models themselves.
This article is based on reporting by Quanta Magazine. Read the original article.
Originally published on quantamagazine.org
