Benchmark Almanac

Chess Puzzles

GPT-6 Astra 76% leads since Sep 2026 Not in model cards

Models find the single best next move in 100 chess puzzles that Epoch generated with a chess engine. Scored by the share solved.

Made by Epoch AI Released Dec 2025 Puzzles and chess All tasks public Last model added Oct 2026 Scores refreshed 31 Oct 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
epoch-research/chess-puzzles-mirror 0 stars Leaderboard epochai.substack.com
83
Models scored in Epoch AI data

GPT-6 Astra leads with 76%

Best score per model in Epoch AI’s data. Where a model ran with several settings or harnesses, we keep its best.

83 models. Scroll sideways for the rest.

  1. 76.0 GPT-6 Astra
  2. 68.0 GPT-5.6 Sol
  3. 68.0 GPT-6.1 Sol
  4. 67.0 Claude Opus 5.5
  5. 66.0 Gemini 3.7 Flash
  6. 63.0 GPT-5.5
  7. 62.0 GPT-5.5 Pro
  8. 60.0 GPT-5.2 Pro
  9. 59.0 GPT-5.4 Pro
  10. 59.0 Claude Sonnet 5.5
  11. 59.0 Gemini 4 Argon
  12. 58.0 Gemini 3.8 Flash
  13. 56.0 GPT-6 Sol
  14. 55.0 Claude Haiku 5.5
  15. 52.0 Claude Opus 4.8
  16. 51.0 Gemini 3.1 Pro
  17. 48.0 GPT-5 Pro
  18. 48.0 GPT-5.2
  19. 48.0 GPT-5.4
  20. 48.0 Claude Fable 5.1
  21. 48.0 DeepSeek V4.1 Flash
  22. 47.0 Claude Fable 5
  23. 47.0 GPT-5.6 Terra
  24. 46.0 Qwen 3.8 Max
  25. 46.0 DeepSeek V4 Pro 0813
  26. 46.0 DeepSeek V4.1 Flash 1017
  27. 45.0 GPT-5.4 Nano
  28. 45.0 Gemini 3.5 Flash
  29. 45.0 Claude Sonnet 5
  30. 43.0 GPT-5.6 Luna
  31. 43.0 Claude Opus 5
  32. 42.0 GPT-5.1
  33. 42.0 Gemini 3.6 Flash
  34. 41.0 Claude Opus 4.7
  35. 41.0 Grok 4.7
  36. 39.0 Gemini 3 Flash
  37. 37.0 o3
  38. 37.0 Kimi K3
  39. 37.0 GLM-5.3
  40. 37.0 Qwen3.8 Max (0902)
  41. 37.0 Muse Spark 1.3
  42. 36.0 GPT-5
  43. 36.0 GPT-6 Luna
  44. 35.0 Grok 4.5
  45. 34.0 Muse Spark 1.1
  46. 33.0 Grok 4.6
  47. 33.0 Mistral Large 4
  48. 32.0 DeepSeek V4 Flash 0731
  49. 29.0 Grok 4.20
  50. 27.0 Muse Spark 1.2
  51. 26.0 Qwen3.7-Max
  52. 24.0 GPT-5 mini
  53. 23.0 Claude Sonnet 4.6
  54. 23.0 GPT-5.4 Mini
  55. 22.0 o4-mini
  56. 22.0 GPT-5 nano
  57. 22.0 GLM-5.2
  58. 21.0 GLM-4.7
  59. 21.0 Claude Opus 4.6
  60. 21.0 Inkling
  61. 21.0 Gemini 3.5 Flash-Lite
  62. 21.0 GLM-5.3-Flash
  63. 20.0 GLM-5.1
  64. 20.0 Kimi K2.6
  65. 19.0 DeepSeek-V4-Pro
  66. 18.0 Grok 4.3 Beta
  67. 15.0 o3-mini
  68. 14.0 Claude Opus 4.5
  69. 14.0 Qwen 3.5 Flash (hosted 35B-A3B)
  70. 14.0 Qwen 3.6 Plus
  71. 14.0 Kimi K2.7 Code
  72. 13.0 MiniMax-M3
  73. 12.0 Qwen 3.6 Flash
  74. 10.0 o1
  75. 10.0 Claude Sonnet 4.5
  76. 8.00 Claude Haiku 4.5
  77. 8.00 Qwen 3.5 Plus (hosted 397B-A17B)
  78. 6.00 GPT-4o
  79. 3.00 Claude 3 Opus
  80. 3.00 GPT-4o mini
  81. 3.00 GPT-4.1
  82. 1.00 GPT-4.1 mini
  83. 0.0 GPT-4.1 nano
  1. 1 GPT-6 Astra pro, max 76%
  2. 2 GPT-5.6 Sol pro, max 68%
  3. 3 GPT-6.1 Sol max 68%
  4. 4 Claude Opus 5.5 max 67%
  5. 5 Gemini 3.7 Flash high 66%
  6. 6 GPT-5.5 xhigh 63%
  7. 7 GPT-5.5 Pro xhigh 62%
  8. 8 GPT-5.2 Pro xhigh 60%
  9. 9 GPT-5.4 Pro xhigh 59%
  10. 10 Claude Sonnet 5.5 max 59%
  11. 11 Gemini 4 Argon high 59%
  12. 12 Gemini 3.8 Flash high 58%
  13. 13 GPT-6 Sol max 56%
  14. 14 Claude Haiku 5.5 max 55%
  15. 15 Claude Opus 4.8 max 52%
  16. 16 Gemini 3.1 Pro high 51%
  17. 17 GPT-5 Pro high 48%
  18. 18 GPT-5.2 xhigh 48%
  19. 19 GPT-5.4 xhigh 48%
  20. 20 Claude Fable 5.1 max 48%
  21. 21 DeepSeek V4.1 Flash max 48%
  22. 22 Claude Fable 5 max 47%
  23. 23 GPT-5.6 Terra max 47%
  24. 24 Qwen 3.8 Max xhigh 46%
  25. 25 DeepSeek V4 Pro 0813 max 46%
  26. 26 DeepSeek V4.1 Flash 1017 max 46%
  27. 27 GPT-5.4 Nano xhigh 45%
  28. 28 Gemini 3.5 Flash high 45%
  29. 29 Claude Sonnet 5 max 45%
  30. 30 GPT-5.6 Luna max 43%
  31. 31 Claude Opus 5 max 43%
  32. 32 GPT-5.1 high 42%
  33. 33 Gemini 3.6 Flash low 42%
  34. 34 Claude Opus 4.7 max 41%
  35. 35 Grok 4.7 xhigh 41%
  36. 36 Gemini 3 Flash high 39%
  37. 37 o3 medium 37%
  38. 38 Kimi K3 high 37%
  39. 39 GLM-5.3 max 37%
  40. 40 Qwen3.8 Max (0902) xhigh 37%
  41. 41 Muse Spark 1.3 max 37%
  42. 42 GPT-5 high 36%
  43. 43 GPT-6 Luna max 36%
  44. 44 Grok 4.5 high 35%
  45. 45 Muse Spark 1.1 xhigh 34%
  46. 46 Grok 4.6 xhigh 33%
  47. 47 Mistral Large 4 high 33%
  48. 48 DeepSeek V4 Flash 0731 max 32%
  49. 49 Grok 4.20 29%
  50. 50 Muse Spark 1.2 xhigh 27%
  51. 51 Qwen3.7-Max 26%
  52. 52 GPT-5 mini high 24%
  53. 53 Claude Sonnet 4.6 max 23%
  54. 54 GPT-5.4 Mini high 23%
  55. 55 o4-mini medium 22%
  56. 56 GPT-5 nano high 22%
  57. 57 GLM-5.2 max 22%
  58. 58 GLM-4.7 21%
  59. 59 Claude Opus 4.6 max 21%
  60. 60 Inkling xhigh 21%
  61. 61 Gemini 3.5 Flash-Lite low 21%
  62. 62 GLM-5.3-Flash max 21%
  63. 63 GLM-5.1 20%
  64. 64 Kimi K2.6 20%
  65. 65 DeepSeek-V4-Pro max 19%
  66. 66 Grok 4.3 Beta high 18%
  67. 67 o3-mini high 15%
  68. 68 Claude Opus 4.5 59k thinking 14%
  69. 69 Qwen 3.5 Flash (hosted 35B-A3B) 14%
  70. 70 Qwen 3.6 Plus 2026-04-02 14%
  71. 71 Kimi K2.7 Code 14%
  72. 72 MiniMax-M3 13%
  73. 73 Qwen 3.6 Flash 2026-04-16 12%
  74. 74 o1 high 10%
  75. 75 Claude Sonnet 4.5 59k thinking 10%
  76. 76 Claude Haiku 4.5 59k thinking 8.00%
  77. 77 Qwen 3.5 Plus (hosted 397B-A17B) 8.00%
  78. 78 GPT-4o Aug 2024 6.00%
  79. 79 Claude 3 Opus 3.00%
  80. 80 GPT-4o mini 3.00%
  81. 81 GPT-4.1 3.00%
  82. 82 GPT-4.1 mini 1.00%
  83. 83 GPT-4.1 nano 0%
#ModelScoreSettingDateSource
1 GPT-6 Astra 76% pro, max Sep 2026 Epoch AI
2 GPT-5.6 Sol 68% pro, max Jul 2026 Epoch AI
3 GPT-6.1 Sol 68% max Sep 2026 Epoch AI
4 Claude Opus 5.5 67% max Sep 2026 Epoch AI
5 Gemini 3.7 Flash 66% high Aug 2026 Epoch AI
6 GPT-5.5 63% xhigh Apr 2026 Epoch AI
7 GPT-5.5 Pro 62% xhigh Apr 2026 Epoch AI
8 GPT-5.2 Pro 60% xhigh Dec 2025 Epoch AI
9 GPT-5.4 Pro 59% xhigh Mar 2026 Epoch AI
10 Claude Sonnet 5.5 59% max Sep 2026 Epoch AI
11 Gemini 4 Argon 59% high Sep 2026 Epoch AI
12 Gemini 3.8 Flash 58% high Sep 2026 Epoch AI
13 GPT-6 Sol 56% max Sep 2026 Epoch AI
14 Claude Haiku 5.5 55% max Oct 2026 Epoch AI
15 Claude Opus 4.8 52% max May 2026 Epoch AI
16 Gemini 3.1 Pro 51% high Feb 2026 Epoch AI
17 GPT-5 Pro 48% high Oct 2025 Epoch AI
18 GPT-5.2 48% xhigh Dec 2025 Epoch AI
19 GPT-5.4 48% xhigh Mar 2026 Epoch AI
20 Claude Fable 5.1 48% max Sep 2026 Epoch AI
21 DeepSeek V4.1 Flash 48% max Sep 2026 Epoch AI
22 Claude Fable 5 47% max Jun 2026 Epoch AI
23 GPT-5.6 Terra 47% max Jul 2026 Epoch AI
24 Qwen 3.8 Max 46% xhigh Aug 2026 Epoch AI
25 DeepSeek V4 Pro 0813 46% max Aug 2026 Epoch AI
26 DeepSeek V4.1 Flash 1017 46% max Oct 2026 Epoch AI
27 GPT-5.4 Nano 45% xhigh Mar 2026 Epoch AI
28 Gemini 3.5 Flash 45% high May 2026 Epoch AI
29 Claude Sonnet 5 45% max Jun 2026 Epoch AI
30 GPT-5.6 Luna 43% max Jul 2026 Epoch AI
31 Claude Opus 5 43% max Jul 2026 Epoch AI
32 GPT-5.1 42% high Nov 2025 Epoch AI
33 Gemini 3.6 Flash 42% low Jul 2026 Epoch AI
34 Claude Opus 4.7 41% max Apr 2026 Epoch AI
35 Grok 4.7 41% xhigh Sep 2026 Epoch AI
36 Gemini 3 Flash 39% high Dec 2025 Epoch AI
37 o3 37% medium Apr 2025 Epoch AI
38 Kimi K3 37% high Jul 2026 Epoch AI
39 GLM-5.3 37% max Aug 2026 Epoch AI
40 Qwen3.8 Max (0902) 37% xhigh Sep 2026 Epoch AI
41 Muse Spark 1.3 37% max Sep 2026 Epoch AI
42 GPT-5 36% high Aug 2025 Epoch AI
43 GPT-6 Luna 36% max Sep 2026 Epoch AI
44 Grok 4.5 35% high Jul 2026 Epoch AI
45 Muse Spark 1.1 34% xhigh Jul 2026 Epoch AI
46 Grok 4.6 33% xhigh Aug 2026 Epoch AI
47 Mistral Large 4 33% high Oct 2026 Epoch AI
48 DeepSeek V4 Flash 0731 32% max Jul 2026 Epoch AI
49 Grok 4.20 29% Feb 2026 Epoch AI
50 Muse Spark 1.2 27% xhigh Aug 2026 Epoch AI
51 Qwen3.7-Max 26% May 2026 Epoch AI
52 GPT-5 mini 24% high Aug 2025 Epoch AI
53 Claude Sonnet 4.6 23% max Feb 2026 Epoch AI
54 GPT-5.4 Mini 23% high Mar 2026 Epoch AI
55 o4-mini 22% medium Apr 2025 Epoch AI
56 GPT-5 nano 22% high Aug 2025 Epoch AI
57 GLM-5.2 22% max Jun 2026 Epoch AI
58 GLM-4.7 21% Dec 2025 Epoch AI
59 Claude Opus 4.6 21% max Feb 2026 Epoch AI
60 Inkling 21% xhigh Jul 2026 Epoch AI
61 Gemini 3.5 Flash-Lite 21% low Jul 2026 Epoch AI
62 GLM-5.3-Flash 21% max Aug 2026 Epoch AI
63 GLM-5.1 20% Apr 2026 Epoch AI
64 Kimi K2.6 20% Apr 2026 Epoch AI
65 DeepSeek-V4-Pro 19% max Apr 2026 Epoch AI
66 Grok 4.3 Beta 18% high Apr 2026 Epoch AI
67 o3-mini 15% high Jan 2025 Epoch AI
68 Claude Opus 4.5 14% 59k thinking Nov 2025 Epoch AI
69 Qwen 3.5 Flash (hosted 35B-A3B) 14% Feb 2026 Epoch AI
70 Qwen 3.6 Plus 14% 2026-04-02 Mar 2026 Epoch AI
71 Kimi K2.7 Code 14% Jun 2026 Epoch AI
72 MiniMax-M3 13% Jun 2026 Epoch AI
73 Qwen 3.6 Flash 12% 2026-04-16 Apr 2026 Epoch AI
74 o1 10% high Dec 2024 Epoch AI
75 Claude Sonnet 4.5 10% 59k thinking Sep 2025 Epoch AI
76 Claude Haiku 4.5 8.00% 59k thinking Oct 2025 Epoch AI
77 Qwen 3.5 Plus (hosted 397B-A17B) 8.00% Feb 2026 Epoch AI
78 GPT-4o Aug 2024 6.00% Aug 2024 Epoch AI
79 Claude 3 Opus 3.00% Feb 2024 Epoch AI
80 GPT-4o mini 3.00% Jul 2024 Epoch AI
81 GPT-4.1 3.00% Apr 2025 Epoch AI
82 GPT-4.1 mini 1.00% Apr 2025 Epoch AI
83 GPT-4.1 nano 0% Apr 2025 Epoch AI

5 models have held number one since its release in Dec 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Epoch AI runs 5 leaders
GPT-6 Astra 76%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Run or collected by Epoch AI Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Random guessing GPT-6 Astra GPT-5.6 Sol GPT-5.5 GPT-5 Pro o3 o3-mini o1 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Random guessing GPT-6 Astra GPT-5.6 Sol

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Games and long tasks

All 68 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 100%
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Scores come from in-game progress or outcomes per title. Nemotron 3 Nano 4B 22.9
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. Gemini 3 Flash 41.9%
  • Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. Measures game progress and harmful actions per 100 steps. Muse Spark 2.40