Benchmark Almanac

Mystery Game Puzzles

GPT-6 Astra 84% leads since Sep 2026 Not in model cards

100 mid-game positions from an undisclosed puzzle variant of a well-known game. Model picks the single best move, checked against an answer key.

Game identity, prompts and positions are kept secret to prevent targeted preparation. Likely the 'game-puzzles' benchmark in the Epoch post.

Made by Epoch AI Released Jul 2026 Puzzles and chess Tasks kept private Last model added Oct 2026 Scores refreshed 31 Oct 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Leaderboard
80
Models scored in Epoch AI data

GPT-6 Astra leads with 84%

Best score per model in Epoch AI’s data. Where a model ran with several settings or harnesses, we keep its best.

80 models. Scroll sideways for the rest.

  1. 84.0 GPT-6 Astra
  2. 80.0 GPT-6.1 Sol
  3. 74.0 GPT-6.1 Terra
  4. 71.0 Claude Opus 5.5
  5. 65.0 Claude Sonnet 5.5
  6. 64.0 Gemini 4 Argon
  7. 59.0 Claude Opus 5
  8. 58.0 GPT-5.6 Sol
  9. 58.0 Claude Fable 5.1
  10. 56.0 GPT-5.5
  11. 56.0 GPT-6 Sol
  12. 52.0 Claude Fable 5
  13. 51.0 Muse Wonder
  14. 49.0 DeepSeek V4.1 Flash 1017
  15. 47.0 Gemini 3.8 Flash
  16. 43.0 DeepSeek V4 Pro 0813
  17. 43.0 DeepSeek V4.1 Flash
  18. 38.0 Qwen 3.8 Max
  19. 37.0 GPT-5.4
  20. 37.0 Gemini 3.7 Flash
  21. 36.0 Claude Opus 4.8
  22. 35.0 Claude Sonnet 5
  23. 35.0 GPT-5.6 Terra
  24. 34.0 Gemini 3.1 Pro
  25. 34.0 DeepSeek V4 Flash 0731
  26. 34.0 Grok 4.6
  27. 33.0 GLM-5.3
  28. 32.0 Gemini 3.5 Flash
  29. 32.0 Qwen3.7-Max
  30. 30.0 Gemini 3.6 Flash
  31. 30.0 Claude Haiku 5.5
  32. 29.0 o3
  33. 29.0 Grok 4.7
  34. 28.0 Claude Opus 4.7
  35. 26.0 Gemini 3 Flash
  36. 26.0 Kimi K3
  37. 25.0 Claude Opus 4.6
  38. 25.0 Muse Spark 1.3
  39. 23.0 GPT-5
  40. 23.0 GPT-5.2
  41. 22.0 Claude Opus 4.5
  42. 22.0 Qwen 3.6 35B-A3B
  43. 21.0 Claude Opus 4.1
  44. 21.0 GPT-5.6 Luna
  45. 20.0 Qwen 3.5 Flash (hosted 35B-A3B)
  46. 20.0 Nemotron 3 Ultra
  47. 19.0 GPT-5.1
  48. 19.0 Qwen 3.6 Max (Preview)
  49. 19.0 GLM-5.2
  50. 19.0 Gemini 3.5 Flash-Lite
  51. 18.0 Qwen3.5 397B-A17B
  52. 18.0 Kimi K2.6
  53. 18.0 Qwen 3.6 Flash
  54. 17.0 Claude Sonnet 4.5
  55. 17.0 Qwen 3.5 Plus (hosted 397B-A17B)
  56. 17.0 Qwen3.5-122B-A10B
  57. 17.0 DeepSeek-V4-Pro
  58. 17.0 Qwen3.7-Plus
  59. 17.0 Mistral Large 4
  60. 16.0 Claude Sonnet 4.6
  61. 15.0 Qwen3.7 Flash
  62. 12.0 GPT-4
  63. 12.0 GPT-4o mini
  64. 12.0 Qwen 3.6 Plus
  65. 11.0 GPT-5.4 Mini
  66. 10.0 GPT-5 mini
  67. 9.00 Qwen3-235B-A22B-Thinking
  68. 9.00 GPT-5 nano
  69. 9.00 GPT-5.4 Nano
  70. 8.00 MiniMax-M3
  71. 8.00 GLM-5.3-Flash
  72. 7.00 o3-mini
  73. 7.00 GPT-4.1 mini
  74. 7.00 Qwen3.6 27B
  75. 7.00 GPT-6 Luna
  76. 6.00 Inkling-Small
  77. 5.00 o4-mini
  78. 5.00 Qwen3-Max
  79. 3.00 GPT-3.5 Turbo
  80. 2.00 gpt-oss-120b
  1. 1 GPT-6 Astra max 84%
  2. 2 GPT-6.1 Sol max 80%
  3. 3 GPT-6.1 Terra max 74%
  4. 4 Claude Opus 5.5 max 71%
  5. 5 Claude Sonnet 5.5 max 65%
  6. 6 Gemini 4 Argon high 64%
  7. 7 Claude Opus 5 max 59%
  8. 8 GPT-5.6 Sol max 58%
  9. 9 Claude Fable 5.1 max 58%
  10. 10 GPT-5.5 xhigh 56%
  11. 11 GPT-6 Sol max 56%
  12. 12 Claude Fable 5 max 52%
  13. 13 Muse Wonder max 51%
  14. 14 DeepSeek V4.1 Flash 1017 max 49%
  15. 15 Gemini 3.8 Flash high 47%
  16. 16 DeepSeek V4 Pro 0813 max 43%
  17. 17 DeepSeek V4.1 Flash max 43%
  18. 18 Qwen 3.8 Max xhigh 38%
  19. 19 GPT-5.4 xhigh 37%
  20. 20 Gemini 3.7 Flash high 37%
  21. 21 Claude Opus 4.8 max 36%
  22. 22 Claude Sonnet 5 max 35%
  23. 23 GPT-5.6 Terra max 35%
  24. 24 Gemini 3.1 Pro high 34%
  25. 25 DeepSeek V4 Flash 0731 max 34%
  26. 26 Grok 4.6 xhigh 34%
  27. 27 GLM-5.3 max 33%
  28. 28 Gemini 3.5 Flash high 32%
  29. 29 Qwen3.7-Max 32%
  30. 30 Gemini 3.6 Flash high 30%
  31. 31 Claude Haiku 5.5 max 30%
  32. 32 o3 high 29%
  33. 33 Grok 4.7 xhigh 29%
  34. 34 Claude Opus 4.7 max 28%
  35. 35 Gemini 3 Flash low 26%
  36. 36 Kimi K3 max 26%
  37. 37 Claude Opus 4.6 max 25%
  38. 38 Muse Spark 1.3 max 25%
  39. 39 GPT-5 high 23%
  40. 40 GPT-5.2 high 23%
  41. 41 Claude Opus 4.5 48k thinking 22%
  42. 42 Qwen 3.6 35B-A3B none 22%
  43. 43 Claude Opus 4.1 24K 21%
  44. 44 GPT-5.6 Luna max 21%
  45. 45 Qwen 3.5 Flash (hosted 35B-A3B) 20%
  46. 46 Nemotron 3 Ultra 20%
  47. 47 GPT-5.1 low 19%
  48. 48 Qwen 3.6 Max (Preview) none 19%
  49. 49 GLM-5.2 low 19%
  50. 50 Gemini 3.5 Flash-Lite low 19%
  51. 51 Qwen3.5 397B-A17B none 18%
  52. 52 Kimi K2.6 18%
  53. 53 Qwen 3.6 Flash 2026-04-16 18%
  54. 54 Claude Sonnet 4.5 48k thinking 17%
  55. 55 Qwen 3.5 Plus (hosted 397B-A17B) none 17%
  56. 56 Qwen3.5-122B-A10B none 17%
  57. 57 DeepSeek-V4-Pro none 17%
  58. 58 Qwen3.7-Plus 17%
  59. 59 Mistral Large 4 high 17%
  60. 60 Claude Sonnet 4.6 low 16%
  61. 61 Qwen3.7 Flash 15%
  62. 62 GPT-4 Jun 2023 12%
  63. 63 GPT-4o mini 12%
  64. 64 Qwen 3.6 Plus 2026-04-02 12%
  65. 65 GPT-5.4 Mini none 11%
  66. 66 GPT-5 mini minimal 10%
  67. 67 Qwen3-235B-A22B-Thinking Jul 2025 9.00%
  68. 68 GPT-5 nano medium 9.00%
  69. 69 GPT-5.4 Nano none 9.00%
  70. 70 MiniMax-M3 none 8.00%
  71. 71 GLM-5.3-Flash max 8.00%
  72. 72 o3-mini high 7.00%
  73. 73 GPT-4.1 mini 7.00%
  74. 74 Qwen3.6 27B none 7.00%
  75. 75 GPT-6 Luna max 7.00%
  76. 76 Inkling-Small xhigh 6.00%
  77. 77 o4-mini high 5.00%
  78. 78 Qwen3-Max 5.00%
  79. 79 GPT-3.5 Turbo Jan 2024 3.00%
  80. 80 gpt-oss-120b medium 2.00%
#ModelScoreSettingDateSource
1 GPT-6 Astra 84% max Sep 2026 Epoch AI
2 GPT-6.1 Sol 80% max Sep 2026 Epoch AI
3 GPT-6.1 Terra 74% max Oct 2026 Epoch AI
4 Claude Opus 5.5 71% max Sep 2026 Epoch AI
5 Claude Sonnet 5.5 65% max Sep 2026 Epoch AI
6 Gemini 4 Argon 64% high Sep 2026 Epoch AI
7 Claude Opus 5 59% max Jul 2026 Epoch AI
8 GPT-5.6 Sol 58% max Jul 2026 Epoch AI
9 Claude Fable 5.1 58% max Sep 2026 Epoch AI
10 GPT-5.5 56% xhigh Apr 2026 Epoch AI
11 GPT-6 Sol 56% max Sep 2026 Epoch AI
12 Claude Fable 5 52% max Jun 2026 Epoch AI
13 Muse Wonder 51% max Oct 2026 Epoch AI
14 DeepSeek V4.1 Flash 1017 49% max Oct 2026 Epoch AI
15 Gemini 3.8 Flash 47% high Sep 2026 Epoch AI
16 DeepSeek V4 Pro 0813 43% max Aug 2026 Epoch AI
17 DeepSeek V4.1 Flash 43% max Sep 2026 Epoch AI
18 Qwen 3.8 Max 38% xhigh Aug 2026 Epoch AI
19 GPT-5.4 37% xhigh Mar 2026 Epoch AI
20 Gemini 3.7 Flash 37% high Aug 2026 Epoch AI
21 Claude Opus 4.8 36% max May 2026 Epoch AI
22 Claude Sonnet 5 35% max Jun 2026 Epoch AI
23 GPT-5.6 Terra 35% max Jul 2026 Epoch AI
24 Gemini 3.1 Pro 34% high Feb 2026 Epoch AI
25 DeepSeek V4 Flash 0731 34% max Jul 2026 Epoch AI
26 Grok 4.6 34% xhigh Aug 2026 Epoch AI
27 GLM-5.3 33% max Aug 2026 Epoch AI
28 Gemini 3.5 Flash 32% high May 2026 Epoch AI
29 Qwen3.7-Max 32% May 2026 Epoch AI
30 Gemini 3.6 Flash 30% high Jul 2026 Epoch AI
31 Claude Haiku 5.5 30% max Oct 2026 Epoch AI
32 o3 29% high Apr 2025 Epoch AI
33 Grok 4.7 29% xhigh Sep 2026 Epoch AI
34 Claude Opus 4.7 28% max Apr 2026 Epoch AI
35 Gemini 3 Flash 26% low Dec 2025 Epoch AI
36 Kimi K3 26% max Jul 2026 Epoch AI
37 Claude Opus 4.6 25% max Feb 2026 Epoch AI
38 Muse Spark 1.3 25% max Sep 2026 Epoch AI
39 GPT-5 23% high Aug 2025 Epoch AI
40 GPT-5.2 23% high Dec 2025 Epoch AI
41 Claude Opus 4.5 22% 48k thinking Nov 2025 Epoch AI
42 Qwen 3.6 35B-A3B 22% none Apr 2026 Epoch AI
43 Claude Opus 4.1 21% 24K Aug 2025 Epoch AI
44 GPT-5.6 Luna 21% max Jul 2026 Epoch AI
45 Qwen 3.5 Flash (hosted 35B-A3B) 20% Feb 2026 Epoch AI
46 Nemotron 3 Ultra 20% Jun 2026 Epoch AI
47 GPT-5.1 19% low Nov 2025 Epoch AI
48 Qwen 3.6 Max (Preview) 19% none Apr 2026 Epoch AI
49 GLM-5.2 19% low Jun 2026 Epoch AI
50 Gemini 3.5 Flash-Lite 19% low Jul 2026 Epoch AI
51 Qwen3.5 397B-A17B 18% none Feb 2026 Epoch AI
52 Kimi K2.6 18% Apr 2026 Epoch AI
53 Qwen 3.6 Flash 18% 2026-04-16 Apr 2026 Epoch AI
54 Claude Sonnet 4.5 17% 48k thinking Sep 2025 Epoch AI
55 Qwen 3.5 Plus (hosted 397B-A17B) 17% none Feb 2026 Epoch AI
56 Qwen3.5-122B-A10B 17% none Feb 2026 Epoch AI
57 DeepSeek-V4-Pro 17% none Apr 2026 Epoch AI
58 Qwen3.7-Plus 17% Jun 2026 Epoch AI
59 Mistral Large 4 17% high Oct 2026 Epoch AI
60 Claude Sonnet 4.6 16% low Feb 2026 Epoch AI
61 Qwen3.7 Flash 15% Jul 2026 Epoch AI
62 GPT-4 Jun 2023 12% Jun 2023 Epoch AI
63 GPT-4o mini 12% Jul 2024 Epoch AI
64 Qwen 3.6 Plus 12% 2026-04-02 Mar 2026 Epoch AI
65 GPT-5.4 Mini 11% none Mar 2026 Epoch AI
66 GPT-5 mini 10% minimal Aug 2025 Epoch AI
67 Qwen3-235B-A22B-Thinking Jul 2025 9.00% Jul 2025 Epoch AI
68 GPT-5 nano 9.00% medium Aug 2025 Epoch AI
69 GPT-5.4 Nano 9.00% none Mar 2026 Epoch AI
70 MiniMax-M3 8.00% none Jun 2026 Epoch AI
71 GLM-5.3-Flash 8.00% max Aug 2026 Epoch AI
72 o3-mini 7.00% high Jan 2025 Epoch AI
73 GPT-4.1 mini 7.00% Apr 2025 Epoch AI
74 Qwen3.6 27B 7.00% none Apr 2026 Epoch AI
75 GPT-6 Luna 7.00% max Sep 2026 Epoch AI
76 Inkling-Small 6.00% xhigh Jul 2026 Epoch AI
77 o4-mini 5.00% high Apr 2025 Epoch AI
78 Qwen3-Max 5.00% Sep 2025 Epoch AI
79 GPT-3.5 Turbo Jan 2024 3.00% Jan 2024 Epoch AI
80 gpt-oss-120b 2.00% medium Aug 2025 Epoch AI

4 models have held number one since its release in Jul 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Epoch AI runs 4 leaders
GPT-6 Astra 84%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Run or collected by Epoch AI Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 2027 Near the ceiling (85%) Random guessing GPT-6 Astra Claude Opus 5 GPT-5.4 o3 GPT-4 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 2027 Near the ceiling (85%) Random guessing GPT-6 Astra Claude Opus 5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Games and long tasks

All 68 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 100%
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Scores come from in-game progress or outcomes per title. Nemotron 3 Nano 4B 22.9
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. Gemini 3 Flash 41.9%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. Measures game progress and harmful actions per 100 steps. Muse Spark 2.40