How this works
We collect scores from lab model cards, official leaderboards and independent runs, and match each one to a single name for its model. We build the site from data files in our GitHub repository.
What gets cataloged
We count a named test of AI models as a benchmark if anyone can run it again. It can be for language models, agents, vision and audio models, robot policies or world models. We add one to the catalog if it passes at least one of four tests.
What stays out: test sets made only to evaluate one paper’s own method, training datasets, surveys, renamed copies of a test we already have, and narrow tasks outside general AI ability, such as crowd counting or defect detection. A new version that changes the tasks or the scoring gets its own record. One that only fixes broken tasks doesn’t.
We've checked and left out 605 finds so far, and the New page lists each one with our reason. A find that fails now can pass later, once a lab reports it or people start using it. To suggest one, use the submission form.
The data
Lab model cards. 479 launch posts from 51 labs, from March 2023 to 2 Nov 2026. For each post we record the benchmarks in its headline results and the model's score on each, as the post gives it. Whether we call a benchmark in use, fading or dropped depends only on these posts. We also read 320 system cards, model cards and technical reports that came out with the releases. From these we record every evaluation each document reports, safety and frontier-risk tests included, along with the section each result appears in. Labs pick their own settings, so card scores don't compare as cleanly as independent runs. We say on each page which kind of score it shows.
Official leaderboards. Many benchmarks keep their own leaderboard, and new models usually show up there first. We import each board from the data file it publishes, using the scripts in scripts/leaderboards/, and match its model names to ours. If a benchmark's own board has five or more models, we lead with it on that benchmark's page.
Independent runs. Scores from Epoch AI Benchmarking Hub (CC BY 4.0), retrieved 31 Oct 2026. Epoch runs some of these itself and collects the rest from public leaderboards. We also use its Epoch Capabilities Index, which puts models on one scale across many benchmarks. We leave out the rows Epoch collected from Artificial Analysis, because Artificial Analysis's terms do not allow reuse in other ranking products.
The catalog. 2,721 benchmarks (2,914 counting each version), each with what it tests, who made it, and links to its code, paper, data and leaderboard. We checked every GitHub link against the GitHub API. The list began with benchmarks from lab cards, Epoch's hub and public collections, and research agents filled in the details from primary sources.
One name per model
We match every score to one record in data/models.json, whatever the source called the model. Each record has the model's name as its lab writes it, its lab and release date, whether its weights are open, and the other spellings it goes by. If a lab shipped several versions under one name, each version gets its own record and a snapshot label, such as GPT-4o (May 2024). Reasoning effort and harness are stored as settings on a score, so GPT-5 at high effort is still GPT-5. We rank a model on a benchmark by its best setting.
We do the same for the 103 labs and for benchmarks. Each has one record with its aliases, so "GPQA-D" in one card and "GPQA Diamond" in another count as one benchmark. Versions don't merge, so ARC-AGI-2 has a record of its own.
The labels
- In use
- At least two model cards reported it in the last six months, or the first card to report it is less than six months old.
- Fading
- Not in use, but a card reported it in the last year.
- Dropped
- No card has reported it for a year.
- Saturated
- The best score has passed 85%. If a dropped benchmark's best score had already passed 85% when the last card reported it, we say labs dropped it after it saturated.
Levels and scoring
Each benchmark gets a level for who could solve its tasks. Model scores play no part in it.
- Everyday
- Most adults, without training.
- School
- Students, from grade school to high school.
- University
- Undergraduates and entry-level professionals.
- Competition
- The strongest students, in olympiad-style contests.
- Expert
- Specialists with years of training, often over hours of work.
- Research frontier
- Researchers, on open problems or tasks that take days.
We also record how each benchmark is scored. Votes and matches rank models against each other, and every other method scores them against a fixed standard.
- Checked automatically
- A program checks the result against an answer key, unit tests or the expected final state.
- Graded by a model
- A language model scores each response, usually against a rubric.
- Graded by people
- Human experts grade or compare the outputs.
- Ranked by votes
- People pick the better of two answers, as in an arena, and the votes are turned into ratings.
- Ranked by matches
- Models play against each other, and the results become ratings.
- Settled by events
- Predictions are scored against what later happens.
Versions
When a benchmark comes out in versions, such as Terminal-Bench 1.0 to 4.0 or PostTrainBench 1.0 to 1.2, we keep one record per version and link them as a family. We never mix scores across versions. A card that reports "PostTrainBench v1.1" counts toward 1.1 only. The benchmarks list shows each family once, at its newest version, and each version's page links to the others.
Who held number one
A model takes number one on a benchmark on the day its score beats every earlier score, and holds it until a later model beats it. If two models tie, the earlier one keeps the spot. We date a score to its model's release, or a lab claim to the card that made it. No score is dated before the benchmark came out. Our charts start in January 2023.
We stop counting time at number one once the best score passes 85%, since a lead means little after that. We also stop if more than 45 days go by with no newer model tested, because then nobody knows who would lead. In that case the count ends on the date of the newest model tested. The first model to take a benchmark past 85% gets a close-out, which we count separately. A new version doesn't end the count for the old one. As long as people still test the old version its race goes on, and ARC-AGI-1 wasn't closed out until long after ARC-AGI-2 came out. Shares of time at number one include only these contested days.
Finding new benchmarks
Every morning we automatically read 19 of the sources on our Sources page. We score each find in two stages, first on whether it is a benchmark at all, then on the evidence that it passes one of the four tests above. Finds that score 8 or more go to the review queue. Finds that score 5 to 7 go on a watch list and drop off after 14 days with no new mention. We skip anything we've already left out, and we never scrape X, because its terms forbid it.
Before a find joins the catalog, a person or a research agent checks it against its own paper, repository or leaderboard. The agents also look for newer versions of benchmarks we already have.
Credits
Scores and the capability index: Epoch AI, under CC BY 4.0. Paper abstracts: arXiv metadata, under CC0, and other papers under the license each one names. Lab marks: LobeHub Icons, under the MIT License, and the labs’ own websites and GitHub or Hugging Face organizations. The marks are trademarks of their owners. Type: Newsreader and Geist, under the SIL Open Font License. Our page designs borrow from the benchmark pages of Epoch AI, Artificial Analysis, Arena and Design Arena.