ResearchClawBench
Agents get the literature and raw data behind a published paper, with the paper hidden, and try to rediscover its results. Expert rubrics with weighted criteria grade the outputs.
40 tasks across 10 scientific domains. The paper evaluates 7 research agents and 17 LLMs. The leaderboard takes community agent submissions.
In the authors’ words
AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation. Expert-curated multimodal rubrics decompose the target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery while leaving room for new discovery. We evaluate seven autonomous research (auto-research) agents under a unified protocol and seventeen native LLMs through the lightweight ResearchHarness. Current systems remain far from reliable re-discovery: the strongest autonomous agent, Claude Code, averages 21.5, and the strongest ResearchHarness LLM, Claude-Opus-4.7, averages 20.7, with an LLM frontier mean of only 26.5. Error analysis shows that failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core. ResearchClawBench provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.
- Level
- Research frontier Researchers, on open problems or tasks that take days
- Scored
- Graded by a model A language model scores each response, usually against a rubric.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
DeepSeek-V4-Pro leads with 38.6 among models released by Aug 2026
Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
The newest model with a score here came out in Aug 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 13 more
- 38.6
DeepSeek-V4-Pro
- 34.9 GPT-5.5
- 34.8 GPT-5.6 Sol
- 28.4
Qwen 3.8 Max
- 22.8 Claude Opus 4.8
- 21.5 Claude Opus 4.6
- 20.7 Claude Opus 4.7
- 20.7 GLM-5.2
- 19.8
MiniMax-M3
- 18.8 GPT-5.4
- 18.7
Qwen3.7-Max
- 18.2 GLM-5.1
- 18.0
Qwen 3.6 Plus
- 18.0 Kimi K2.6
- 17.9
Gemini 3.5 Flash
- 16.9 MiMo-V2.5
- 15.3 MiMo-V2-Pro
- 14.2
Qwen3.5 397B-A17B
- 14.0 Kimi K2.5
- 13.5 Grok 4.1
- 13.3
Gemini 3.1 Pro
- 12.9
Tencent Hy3 preview
- 12.4 Grok 4.3
- 1
DeepSeek-V4-Pro 38.6
- 2 GPT-5.5 34.9
- 3 GPT-5.6 Sol 34.8
- 4
Qwen 3.8 Max 28.4
- 5 Claude Opus 4.8 22.8
- 6 Claude Opus 4.6 21.5
- 7 Claude Opus 4.7 20.7
- 8 GLM-5.2 20.7
- 9
MiniMax-M3 19.8
- 10 GPT-5.4 18.8
- 11
Qwen3.7-Max 18.7
- 12 GLM-5.1 18.2
- 13
Qwen 3.6 Plus 18.0
- 14 Kimi K2.6 18.0
- 15
Gemini 3.5 Flash 17.9
- 16 MiMo-V2.5 16.9
- 17 MiMo-V2-Pro 15.3
- 18
Qwen3.5 397B-A17B 14.2
- 19 Kimi K2.5 14.0
- 20 Grok 4.1 13.5
- 21
Gemini 3.1 Pro 13.3
- 22
Tencent Hy3 preview 12.9
- 23 Grok 4.3 12.4
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 38.6 | Qiushi Engine | Apr 2026 | Official leaderboard |
| 2 | GPT-5.5 | 34.9 | InnoClaw (2.0) | Apr 2026 | Official leaderboard |
| 3 | GPT-5.6 Sol | 34.8 | Ligase | Jul 2026 | Official leaderboard |
| 4 | | 28.4 | jiutian-DeepInsight | Aug 2026 | Official leaderboard |
| 5 | Claude Opus 4.8 | 22.8 | Open Science | May 2026 | Official leaderboard |
| 6 | Claude Opus 4.6 | 21.5 | Claude Code | Feb 2026 | Official leaderboard |
| 7 | Claude Opus 4.7 | 20.7 | ResearchHarness | Apr 2026 | Official leaderboard |
| 8 | GLM-5.2 | 20.7 | ResearchHarness | Jun 2026 | Official leaderboard |
| 9 | | 19.8 | ResearchHarness | Jun 2026 | Official leaderboard |
| 10 | GPT-5.4 | 18.8 | EvoScientist (0.1.1) | Mar 2026 | Official leaderboard |
| 11 | | 18.7 | ResearchHarness | May 2026 | Official leaderboard |
| 12 | GLM-5.1 | 18.2 | ResearchHarness | Apr 2026 | Official leaderboard |
| 13 | | 18.0 | ResearchHarness | Mar 2026 | Official leaderboard |
| 14 | Kimi K2.6 | 18.0 | ResearchHarness | Apr 2026 | Official leaderboard |
| 15 | | 17.9 | ResearchHarness | May 2026 | Official leaderboard |
| 16 | MiMo-V2.5 | 16.9 | ResearchHarness | Apr 2026 | Official leaderboard |
| 17 | MiMo-V2-Pro | 15.3 | ResearchHarness | Mar 2026 | Official leaderboard |
| 18 | | 14.2 | ResearchHarness | Feb 2026 | Official leaderboard |
| 19 | Kimi K2.5 | 14.0 | ResearchHarness | Jan 2026 | Official leaderboard |
| 20 | Grok 4.1 | 13.5 | ResearchHarness | Nov 2025 | Official leaderboard |
| 21 | | 13.3 | ResearchHarness | Feb 2026 | Official leaderboard |
| 22 | | 12.9 | ResearchHarness | Apr 2026 | Official leaderboard |
| 23 | Grok 4.3 | 12.4 | ResearchHarness | Apr 2026 | Official leaderboard |
The best score, 38.6, costs $7.63 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| | Qiushi Engine | 38.6 | $7.63 | ||
| GPT-5.5 | InnoClaw (2.0) | 34.9 | $8.92 | ||
| GPT-5.6 Sol | Ligase | 34.8 | $7.65 | ||
| GPT-5.6 Sol | AutoSciRub | 33.2 | $7.85 | ||
| GPT-5.5 | EvoLab | 32.7 | $6.45 | ||
| GPT-5.5 | InnoClaw | 28.8 | $7.10 | ||
| | jiutian-DeepInsight | 28.4 | $10 | ||
| Claude Opus 4.8 | Open Science | 22.8 | $3.44 | ||
| Claude Opus 4.6 | Claude Code | 21.5 | $5.25 | ||
| Claude Opus 4.8 | ResearchHarness | 21.1 | $3.97 | ||
| Claude Opus 4.7 | ResearchHarness | 20.7 | $5.06 | ||
| GLM-5.2 | ResearchHarness | 20.7 | $1.24 | ||
| Claude Opus 4.6 | ResearchHarness | 19.9 | $6.05 | ||
| | ResearchHarness | 19.8 | $0.45 | ||
| GPT-5.4 | EvoScientist (0.1.1) | 18.8 | $4.08 | ||
| | ResearchHarness | 18.7 | $0.42 | ||
| GPT-5.4 | Codex CLI | 18.4 | $2.01 | ||
| GLM-5.1 | ResearchHarness | 18.2 | $1.30 | ||
| | ResearchHarness | 18.0 | $0.52 | ||
| Kimi K2.6 | ResearchHarness | 18.0 | $1.19 | ||
| | ResearchHarness | 17.9 | $1.20 | ||
| | ResearchHarness | 17.1 | $0.28 | ||
| GPT-5.5 | ResearchHarness | 17.0 | $1.82 | ||
| MiMo-V2.5 | ResearchHarness | 16.9 | $0.23 | ||
| GPT-5.4 | OpenClaw | 16.6 | $0.69 | ||
| GPT-5.4 | ResearchClaw | 16.3 | $0.82 | ||
| GPT-5.4 | EvoScientist (0.0.4) | 15.5 | $1.19 | ||
| MiMo-V2-Pro | ResearchHarness | 15.3 | $0.62 | ||
| GPT-5.4 | ResearchHarness | 15.3 | $2.13 | ||
| | ResearchHarness | 14.2 | $0.58 | ||
| Kimi K2.5 | ResearchHarness | 14.0 | $0.51 | ||
| GPT-5.4 | ARIS Codex | 13.6 | $0.74 | ||
| Grok 4.1 | ResearchHarness | 13.5 | $0.04 | ||
| | ResearchHarness | 13.3 | $1.60 | ||
| | ResearchHarness | 12.9 | $0.03 | ||
| GPT-5.4 | Nanobot | 12.8 | $0.49 | ||
| Grok 4.3 | ResearchHarness | 12.4 | $0.44 |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
No lab model card reports it yet
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.
More in Science and reasoning
- GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics.
Gemini 4 Argon 96.4%
- Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
- FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics.
Intern-S2-397B 87%
- BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Opus 5 90.1%
- HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels.
Gemini 3 Pro 78.3%
- LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%