Benchmark Almanac

RSI-Exam

Claude Opus 5.5 53.6% leads since Sep 2026 Not in model cards

Agents get a weak method on a long research task and must turn it into one that scores better on hidden data. It measures recursive self-improvement on executable research.

35 public tasks with grading containers and 53 held back. Runs on Harbor with a sealed network policy. About 190 GitHub stars.

Made by AIMING Lab (UNC Chapel Hill) Released Aug 2026 AI research tasks Public tasks plus a private test set Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
aiming-lab/RSI-Exam 192 stars Dataset Leaderboard rsi-exam.ai
19
Entries on its own leaderboard, 18 models we track

Claude Opus 5.5 leads with 53.6%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

19 entries. Scroll sideways for the rest.

  1. 53.6 Claude Opus 5.5
  2. 51.3 GPT-6 Astra
  3. 49.4 Gemini 4 Argon
  4. 48.1 Claude Fable 5.1
  5. 46.4 Claude Opus 5
  6. 43.3 GPT-5.6 Sol
  7. 40.3 GLM-5.3
  8. 39.2 Qwen3.8 Max (0902)
  9. 39.1 Muse Spark 1.3
  10. 38.4 Grok 4.7
  11. 38.2 Kimi K3
  12. 36.7 Grok 4.6
  13. 35.2 Step 5 Preview
  14. 34.1 Gemini 3.8 Flash
  15. 33.1 GPT-5.5
  16. 32.4 Seed Evolving-0909
  17. 32.3 DeepSeek-V4-Pro
  18. 32.2 Qwen 3.8 Max
  19. 30.9 Gemini 3.7 Flash
  1. 1 Claude Opus 5.5 max 53.6%
  2. 2 GPT-6 Astra max 51.3%
  3. 3 Gemini 4 Argon high 49.4%
  4. 4 Claude Fable 5.1 max 48.1%
  5. 5 Claude Opus 5 max 46.4%
  6. 6 GPT-5.6 Sol max 43.3%
  7. 7 GLM-5.3 max 40.3%
  8. 8 Qwen3.8 Max (0902) xhigh 39.2%
  9. 9 Muse Spark 1.3 max 39.1%
  10. 10 Grok 4.7 xhigh 38.4%
  11. 11 Kimi K3 max 38.2%
  12. 12 Grok 4.6 xhigh 36.7%
  13. 13 Step 5 Preview max 35.2%
  14. 14 Gemini 3.8 Flash high 34.1%
  15. 15 GPT-5.5 xhigh 33.1%
  16. 16 Seed Evolving-0909 max 32.4%
  17. 17 DeepSeek-V4-Pro max 32.3%
  18. 18 Qwen 3.8 Max xhigh 32.2%
  19. 19 Gemini 3.7 Flash high 30.9%
#EntryScoreSettingDateSource
1 Claude Opus 5.5 53.6% max, claude code Sep 2026 Official leaderboard
2 GPT-6 Astra 51.3% max, codex Sep 2026 Official leaderboard
3 Gemini 4 Argon 49.4% high, antigravity Sep 2026 Official leaderboard
4 Claude Fable 5.1 48.1% max, claude code Sep 2026 Official leaderboard
5 Claude Opus 5 46.4% max, claude code Jul 2026 Official leaderboard
6 GPT-5.6 Sol 43.3% max, codex Jul 2026 Official leaderboard
7 GLM-5.3 40.3% max, claude code Aug 2026 Official leaderboard
8 Qwen3.8 Max (0902) 39.2% xhigh, claude code Sep 2026 Official leaderboard
9 Muse Spark 1.3 39.1% max, musecode Sep 2026 Official leaderboard
10 Grok 4.7 38.4% xhigh, grok Sep 2026 Official leaderboard
11 Kimi K3 38.2% max, kimi code Jul 2026 Official leaderboard
12 Grok 4.6 36.7% xhigh, grok Aug 2026 Official leaderboard
13 Step 5 Preview 35.2% max, claude code Sep 2026 Official leaderboard
14 Gemini 3.8 Flash 34.1% high, antigravity Sep 2026 Official leaderboard
15 GPT-5.5 33.1% xhigh, codex Apr 2026 Official leaderboard
16 Seed Evolving-0909 32.4% max, claude code Official leaderboard
17 DeepSeek-V4-Pro 32.3% max, claude code Apr 2026 Official leaderboard
18 Qwen 3.8 Max 32.2% xhigh, qwen coder Aug 2026 Official leaderboard
19 Gemini 3.7 Flash 30.9% high, antigravity Aug 2026 Official leaderboard

The best score, 53.6%, costs $21 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Moonshot AI Z.ai Qwen Google DeepMind Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% $1.00 $2.00 $5.00 $10 $20 $50 $100 Cost per task, log scale Claude Opus 5.5, max GLM-5.3, max Qwen3.8 Max (0902), xhigh Gemini 3.8 Flash, high 0% 20% 40% 60% $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, max GLM-5.3, max
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 max claude code 53.6% $21 280,471
GPT-6 Astra max codex 51.3% $53 340,814
Gemini 4 Argon high antigravity 49.4%
Claude Fable 5.1 max claude code 48.1% $29 230,270
Claude Opus 5 max claude code 46.4% $42 287,844
GPT-5.6 Sol max codex 43.3% $53 233,916
GLM-5.3 max claude code 40.3% $18 273,129
Qwen3.8 Max (0902) xhigh claude code 39.2% $12 327,964
Muse Spark 1.3 max musecode 39.1% $12 340,759
Grok 4.7 xhigh grok 38.4% $13 191,762
Kimi K3 max kimi code 38.2% $10 133,843
Grok 4.6 xhigh grok 36.7% $14 251,550
Step 5 Preview max claude code 35.2%
Gemini 3.8 Flash high antigravity 34.1% $5.22 203,554
GPT-5.5 xhigh codex 33.1% $13 69,741
DeepSeek-V4-Pro max claude code 32.3% $6.66 507,998
Qwen 3.8 Max xhigh qwen coder 32.2% $20 229,019
Gemini 3.7 Flash high antigravity 30.9% $9.34 395,387

4 models have held number one since its release in Aug 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard 4 leaders
Claude Opus 5.5 53.6%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Fable 5.1 GPT-5.6 Sol GPT-5.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Opus 5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in AI research

All 83 in AI research
  • MLS-Bench In use Agent must invent one algorithmic change in an ML research scaffold whose gain transfers across settings. Scored relative to human baselines. Claude Fable 5.1 50.3%
  • PaperBench In use Agent replicates ICML 2024 papers from scratch. LLM judge grades against author-written hierarchical rubrics. Qwen 3.8 Max 93%
  • MLE-bench In use Kaggle-style ML engineering competitions. Agent trains models and is scored against competition leaderboards. GPT-6 Astra 93.8%
  • PostTrainBench 1.2 In use CLI agent gets a base model, one H100 and a time budget to post-train it for a target benchmark. Scored on normalized gain. Claude Fable 5.1 44.6%
  • KernelBench In use Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. MiniMax-M3 28.8%
  • OpenAI PRs Only in system cards Agents replicate internal OpenAI pull requests from a pre-PR branch and a prompt. Graded by hidden unit tests as pass@1. GPT-5.2 55%