Benchmark Almanac

ResearchClawBench

DeepSeek-V4-Pro 38.6 leads since May 2026 Not in model cards

Agents get the literature and raw data behind a published paper, with the paper hidden, and try to rediscover its results. Expert rubrics with weighted criteria grade the outputs.

40 tasks across 10 scientific domains. The paper evaluates 7 research agents and 17 LLMs. The leaderboard takes community agent submissions.

In the authors’ words

AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation. Expert-curated multimodal rubrics decompose the target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery while leaving room for new discovery. We evaluate seven autonomous research (auto-research) agents under a unified protocol and seventeen native LLMs through the lightweight ResearchHarness. Current systems remain far from reliable re-discovery: the strongest autonomous agent, Claude Code, averages 21.5, and the strongest ResearchHarness LLM, Claude-Opus-4.7, averages 20.7, with an LLM frontier mean of only 26.5. Error analysis shows that failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core. ResearchClawBench provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.

Abstract of ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research (arXiv:2606.07591, CC0)
Made by Intern Science (Shanghai AI Laboratory) Released May 2026 Research science All tasks public Last model added Aug 2026 Scores refreshed 2 Nov 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
23
Models scored on its own leaderboard

DeepSeek-V4-Pro leads with 38.6 among models released by Aug 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Aug 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 DeepSeek-V4-Pro 38.6
  2. 2 GPT-5.5 34.9
  3. 3 GPT-5.6 Sol 34.8
  4. 4 Qwen 3.8 Max 28.4
  5. 5 Claude Opus 4.8 22.8
  6. 6 Claude Opus 4.6 21.5
  7. 7 Claude Opus 4.7 20.7
  8. 8 GLM-5.2 20.7
  9. 9 MiniMax-M3 19.8
  10. 10 GPT-5.4 18.8
  11. 11 Qwen3.7-Max 18.7
  12. 12 GLM-5.1 18.2
  13. 13 Qwen 3.6 Plus 18.0
  14. 14 Kimi K2.6 18.0
  15. 15 Gemini 3.5 Flash 17.9
  16. 16 MiMo-V2.5 16.9
  17. 17 MiMo-V2-Pro 15.3
  18. 18 Qwen3.5 397B-A17B 14.2
  19. 19 Kimi K2.5 14.0
  20. 20 Grok 4.1 13.5
  21. 21 Gemini 3.1 Pro 13.3
  22. 22 Tencent Hy3 preview 12.9
  23. 23 Grok 4.3 12.4
#ModelScoreSettingDateSource
1 DeepSeek-V4-Pro 38.6 Qiushi Engine Apr 2026 Official leaderboard
2 GPT-5.5 34.9 InnoClaw (2.0) Apr 2026 Official leaderboard
3 GPT-5.6 Sol 34.8 Ligase Jul 2026 Official leaderboard
4 Qwen 3.8 Max 28.4 jiutian-DeepInsight Aug 2026 Official leaderboard
5 Claude Opus 4.8 22.8 Open Science May 2026 Official leaderboard
6 Claude Opus 4.6 21.5 Claude Code Feb 2026 Official leaderboard
7 Claude Opus 4.7 20.7 ResearchHarness Apr 2026 Official leaderboard
8 GLM-5.2 20.7 ResearchHarness Jun 2026 Official leaderboard
9 MiniMax-M3 19.8 ResearchHarness Jun 2026 Official leaderboard
10 GPT-5.4 18.8 EvoScientist (0.1.1) Mar 2026 Official leaderboard
11 Qwen3.7-Max 18.7 ResearchHarness May 2026 Official leaderboard
12 GLM-5.1 18.2 ResearchHarness Apr 2026 Official leaderboard
13 Qwen 3.6 Plus 18.0 ResearchHarness Mar 2026 Official leaderboard
14 Kimi K2.6 18.0 ResearchHarness Apr 2026 Official leaderboard
15 Gemini 3.5 Flash 17.9 ResearchHarness May 2026 Official leaderboard
16 MiMo-V2.5 16.9 ResearchHarness Apr 2026 Official leaderboard
17 MiMo-V2-Pro 15.3 ResearchHarness Mar 2026 Official leaderboard
18 Qwen3.5 397B-A17B 14.2 ResearchHarness Feb 2026 Official leaderboard
19 Kimi K2.5 14.0 ResearchHarness Jan 2026 Official leaderboard
20 Grok 4.1 13.5 ResearchHarness Nov 2025 Official leaderboard
21 Gemini 3.1 Pro 13.3 ResearchHarness Feb 2026 Official leaderboard
22 Tencent Hy3 preview 12.9 ResearchHarness Apr 2026 Official leaderboard
23 Grok 4.3 12.4 ResearchHarness Apr 2026 Official leaderboard

The best score, 38.6, costs $7.63 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

xAI OpenAI Xiaomi Corp Tencent DeepSeek Qwen Anthropic MiniMax Z.ai Cost frontier One model at each effort level
0.00 10.0 20.0 30.0 40.0 $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale DeepSeek-V4-Pro GPT-5.5 Claude Opus 4.8 GLM-5.2 MiniMax-M3 DeepSeek-V4-Pro Grok 4.1 0.00 10.0 20.0 30.0 40.0 $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale DeepSeek-V4-Pro GPT-5.5
ModelEffortHarnessScoreCost per taskOutput tokens
DeepSeek-V4-Pro Qiushi Engine 38.6 $7.63
GPT-5.5 InnoClaw (2.0) 34.9 $8.92
GPT-5.6 Sol Ligase 34.8 $7.65
GPT-5.6 Sol AutoSciRub 33.2 $7.85
GPT-5.5 EvoLab 32.7 $6.45
GPT-5.5 InnoClaw 28.8 $7.10
Qwen 3.8 Max jiutian-DeepInsight 28.4 $10
Claude Opus 4.8 Open Science 22.8 $3.44
Claude Opus 4.6 Claude Code 21.5 $5.25
Claude Opus 4.8 ResearchHarness 21.1 $3.97
Claude Opus 4.7 ResearchHarness 20.7 $5.06
GLM-5.2 ResearchHarness 20.7 $1.24
Claude Opus 4.6 ResearchHarness 19.9 $6.05
MiniMax-M3 ResearchHarness 19.8 $0.45
GPT-5.4 EvoScientist (0.1.1) 18.8 $4.08
Qwen3.7-Max ResearchHarness 18.7 $0.42
GPT-5.4 Codex CLI 18.4 $2.01
GLM-5.1 ResearchHarness 18.2 $1.30
Qwen 3.6 Plus ResearchHarness 18.0 $0.52
Kimi K2.6 ResearchHarness 18.0 $1.19
Gemini 3.5 Flash ResearchHarness 17.9 $1.20
DeepSeek-V4-Pro ResearchHarness 17.1 $0.28
GPT-5.5 ResearchHarness 17.0 $1.82
MiMo-V2.5 ResearchHarness 16.9 $0.23
GPT-5.4 OpenClaw 16.6 $0.69
GPT-5.4 ResearchClaw 16.3 $0.82
GPT-5.4 EvoScientist (0.0.4) 15.5 $1.19
MiMo-V2-Pro ResearchHarness 15.3 $0.62
GPT-5.4 ResearchHarness 15.3 $2.13
Qwen3.5 397B-A17B ResearchHarness 14.2 $0.58
Kimi K2.5 ResearchHarness 14.0 $0.51
GPT-5.4 ARIS Codex 13.6 $0.74
Grok 4.1 ResearchHarness 13.5 $0.04
Gemini 3.1 Pro ResearchHarness 13.3 $1.60
Tencent Hy3 preview ResearchHarness 12.9 $0.03
GPT-5.4 Nanobot 12.8 $0.49
Grok 4.3 ResearchHarness 12.4 $0.44

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

DeepSeek
Own leaderboard Not tested since Aug 2026

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
10.0 20.0 30.0 40.0 2026 10.0 20.0 30.0 40.0 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Science and reasoning

All 88 in Science and reasoning
  • GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics. Gemini 4 Argon 96.4%
  • Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
  • FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics. Intern-S2-397B 87%
  • BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Opus 5 90.1%
  • HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels. Gemini 3 Pro 78.3%
  • LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%