PHYBench
Models solve 500 original physics problems from high school to Physics Olympiad level. Answers are expressions checked against the key.
At release Gemini 2.5 Pro scored 36.9% against 61.9% for human experts. The paper also adds an expression edit distance score.
In the authors’ words
Current benchmarks for evaluating the reasoning capabilities of Large Language Models (LLMs) face significant limitations: task oversimplification, data contamination, and flawed evaluation items. These deficiencies necessitate more rigorous assessment methods. To address these limitations, we introduce PHYBench, a benchmark of 500 original physics problems ranging from high school to Physics Olympiad difficulty. PHYBench addresses data contamination through original content and employs a systematic curation pipeline to eliminate flawed items. Evaluations show that PHYBench activates more tokens and provides stronger differentiation between reasoning models compared to other baselines like AIME 2024, OlympiadBench and GPQA. Even the best-performing model, Gemini 2.5 Pro, achieves only 36.9% accuracy compared to human experts' 61.9%. To further enhance evaluation precision, we introduce the Expression Edit Distance (EED) Score for mathematical expression assessment, which improves sample efficiency by 204% over binary scoring. Moreover, PHYBench effectively elicits multi-step and multi-condition reasoning, providing a platform for examining models' reasoning robustness, preferences, and deficiencies. The benchmark results and dataset are publicly available at https://www.phybench.cn/.
- Level
- Competition The strongest students, in olympiad-style contests
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Short answer A number, word or short string matched to a key.
Tencent Hy3 leads with 77.4% among models released by Jul 2026
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
The newest model with a score here came out in Jul 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 1
Tencent Hy3 77.4%
- 2
Seed2.0 Pro 74%
- 3
Ring-1T 42.6%
- 4
Seed1.8 41%
- 5
Ministral 3 14B 26%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 77.4% | Jul 2026 | Model card | |
| 2 | | 74% | Feb 2026 | Model card | |
| 3 | | 42.6% | Oct 2025 | Model card | |
| 4 | | 41% | Dec 2025 | Model card | |
| 5 | | 26% | Jan 2026 | Model card |
The benchmark's own leaderboard has 28 models. We plot every source in the charts below.
Scores at each effort level
Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| | thinking | 50.4% | |||
| | 45.8% | ||||
| Grok 4 | 42.3% | ||||
| GPT-5 | 40.4% | ||||
| | 36.9% | ||||
| | 35% | ||||
| o3 | high | 34.8% | |||
| | thinking | 34.5% | |||
| | 33.5% | ||||
| | 33.5% | ||||
| o4-mini | high | 29.4% | |||
| | 28.7% | ||||
| GLM-4.5 | 26% | ||||
| o3-mini | high | 25% | |||
| o4-mini | 24.9% | ||||
| | 22.1% | ||||
| o3-mini | 21.3% | ||||
| Kimi K2 Jul 2025 | 21.3% | ||||
| Grok 3 | 21.2% | ||||
| Claude Sonnet 4 | thinking | 19.2% | |||
| | thinking | 18.2% | |||
| o1 | 18% | ||||
| | 16.6% | ||||
| | 15.8% | ||||
| Claude 3.7 Sonnet | thinking | 15.3% | |||
| o3-mini | low | 13.7% | |||
| | 13.6% | ||||
| GPT-4.1 | 13.2% | ||||
| Claude 3.7 Sonnet | 12.9% | ||||
| | 11.7% | ||||
| GPT-4o Mar 2025 | 6.97% | ||||
| | 6.27% |
3 models have held number one since its release in Apr 2025
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Jul 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 2 Jul 2026 | 77.4 | PHYBench | |
| 14 Feb 2026 | 74 | Phybench | |
| 18 Dec 2025 | 41 | PHYBench |
4 system cards report it, from 3 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 21 Mar 2026 | | 41 | PHYBench (Table 1) | Capabilities |
| 14 Feb 2026 | | 74 | Phybench (Table 3) | Capabilities |
| 13 Jan 2026 | | 26 | PhyBench, reported as pass@16 (Table 5) | Capabilities |
| 21 Oct 2025 | | 42.6 | Phybench (Table 1) | Capabilities |
More in Science and reasoning
- GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics.
Gemini 4 Argon 96.4%
- Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
- FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics.
Intern-S2-397B 87%
- BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Opus 5 90.1%
- HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels.
Gemini 3 Pro 78.3%
- LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%