Benchmark Almanac

Wer wird Millionär? LLM Benchmark

GPT-6 Astra 644444.0 leads since Sep 2026 Not in model cards

Models play 45 games of the German version of Who Wants to Be a Millionaire, each with 15 questions of rising difficulty. Early questions lean on German wordplay and idioms.

Still updated in 2026-10. About 50 GitHub stars, but the Reddit posts reached 759 and 246 points.

Made by ikiruneo (GitHub user) Released Sep 2025 Knowledge in many languages All tasks public Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
ikiruneo/millionaire-bench 50 stars Leaderboard
96
Entries on its own leaderboard, 76 models we track

GPT-6 Astra leads with 644444.0 among models released by Sep 2026

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

96 entries. Scroll sideways for the rest.

  1. 644444.0 GPT-6 Astra
  2. 588889.0 GPT-5.5
  3. 433333.0 Gemini 3.5 Flash
  4. 416667.0 GPT-5.6 Luna
  5. 366667.0 GPT-5
  6. 358333.0 GPT-5.6 Sol
  7. 333333.0 GPT-5.2
  8. 291667.0 GPT-5.1-Codex
  9. 275000.0 GPT-5.4
  10. 266667.0 Gemini 2.5 Pro
  11. 225000.0 GPT-5.4 Mini
  12. 200000.0 GPT-5.2 Codex
  13. 183333.0 GPT-5.6 Terra
  14. 175000.0 DeepSeek V4.1 Flash
  15. 114156.0 GPT-5.1
  16. 114156.0 Gemini 3.1 Flash-Lite
  17. 114156.0 Kimi K2.6
  18. 112800.0 o3
  19. 97889.0 GPT-5 mini
  20. 91111.0 GLM-5.2
  21. 87044.0 Qwen3.7-Max
  22. 76200.0 Grok 4.3
  23. 73489.0 DeepSeek-V4-Pro
  24. 61867.0 DeepSeek-V4-Flash
  25. 56178.0 o4-mini
  26. 27733.0 GPT-5.4 Nano
  27. 27022.0 MiniMax-M2.7
  28. 26311.0 Gemma 4 31B IT
  29. 16000.0 GLM-4.5
  30. 15822.0 Qwen3-235B-A22B
  31. 14756.0 MiniMax-M3
  32. 13156.0 GPT-5 nano
  33. 12622.0 GPT-4o
  34. 10667.0 Gemma 4 26B A4B
  35. 9778.0 MiMo-V2.5
  36. 7022.0 GPT-4.1
  37. 5689.0 Gemini 2.5 Flash
  38. 3956.0 sonar
  39. 3600.0 DeepSeek-V3
  40. 3111.0 gpt-oss-120b
  41. 2800.0 Cohere Command A
  42. 2711.0 Kimi K2
  43. 2489.0 GPT-4.1 mini
  44. 2178.0 QwQ-32B
  45. 1867.0 GLM-4.5-Air
  46. 1667.0 qwen3-235b-a22b-2507
  47. 1422.0 Llama 4 Maverick
  48. 1267.0 GPT-4o mini
  49. 1111.0 DeepSeek-V3.1
  50. 967.0 nvidia-nemotron-3-nano
  51. 933.0 Mistral Small 3.2
  52. 700.0 Gemini 2.5 Flash-Lite
  53. 656.0 qwen3-30b-a3b-2507
  54. 611.0 behemoth-123B-v1.2
  55. 451.0 qwen3-coder-480b
  56. 389.0 GPT-4.1 nano
  57. 353.0 Llama 3.3 70B Instruct
  58. 349.0 Phi-4
  59. 300.0 Ministral 3 14B
  60. 296.0 hermes-4-14b
  61. 296.0 gpt-oss-20b
  62. 289.0 Gemma 3 27B
  63. 280.0 glm-4.6-flash
  64. 244.0 Gemma 3 12B
  65. 238.0 Qwen3-4B-Thinking-2507
  66. 224.0 Mistral NeMo
  67. 211.0 Ministral 3 8B Instruct
  68. 191.0 glm-4.7-flash
  69. 191.0 Gemma 4 12B
  70. 184.0 Ling-flash-2.0
  71. 169.0 Qwen3.5-9B
  72. 156.0 granite-3.2-8b
  73. 149.0 apertus:8b
  74. 149.0 Granite 4.0 H Small
  75. 124.0 k2-horizon-7b-mlx
  76. 118.0 Llama 3.1-8B
  77. 111.0 Gemma 3n E4B
  78. 109.0 qwen3-4b-instruct-2507
  79. 88.0 LFM2-8B-A1B
  80. 86.0 lfm2-2.6b
  81. 84.0 Qwen3-1.7B
  82. 80.0 lfm2-24b-a2b
  83. 79.0 nemotron-nano-9b-v2
  84. 77.0 eurollm-9b-instruct
  85. 77.0 Qwen3.5-4B
  86. 76.0 granite-4-h-tiny
  87. 70.0 Gemma 4 E4B
  88. 67.0 jamba-reasoning-3b
  89. 64.0 Gemma 3 4B
  90. 62.0 ling-mini-2.0
  91. 58.0 Phi-4 Mini
  92. 57.0 Ministral 3 3B Instruct
  93. 53.0 granite-4.0-h-micro
  94. 50.0 Llama 3.2 3B
  95. 36.0 olmo-3-7b-instruct
  96. 32.0 lfm2.5-1.2b
  1. 1 GPT-6 Astra high 644444.0
  2. 2 GPT-5.5 xhigh 588889.0
  3. 3 Gemini 3.5 Flash 433333.0
  4. 4 GPT-5.6 Luna max 416667.0
  5. 5 GPT-5 medium 366667.0
  6. 6 GPT-5.6 Sol max 358333.0
  7. 7 GPT-5.2 medium 333333.0
  8. 8 GPT-5.1-Codex medium 291667.0
  9. 9 GPT-5.4 xhigh 275000.0
  10. 10 Gemini 2.5 Pro Jun 2025 266667.0
  11. 11 GPT-5.4 Mini 225000.0
  12. 12 GPT-5.2 Codex medium 200000.0
  13. 13 GPT-5.6 Terra max 183333.0
  14. 14 DeepSeek V4.1 Flash 175000.0
  15. 15 GPT-5.1 medium 114156.0
  16. 16 Gemini 3.1 Flash-Lite 114156.0
  17. 17 Kimi K2.6 114156.0
  18. 18 o3 medium 112800.0
  19. 19 GPT-5 mini high 97889.0
  20. 20 GLM-5.2 91111.0
  21. 21 Qwen3.7-Max 87044.0
  22. 22 Grok 4.3 high 76200.0
  23. 23 DeepSeek-V4-Pro 73489.0
  24. 24 DeepSeek-V4-Flash 61867.0
  25. 25 o4-mini medium 56178.0
  26. 26 GPT-5.4 Nano xhigh 27733.0
  27. 27 MiniMax-M2.7 27022.0
  28. 28 Gemma 4 31B IT 26311.0
  29. 29 GLM-4.5 16000.0
  30. 30 Qwen3-235B-A22B 15822.0
  31. 31 MiniMax-M3 14756.0
  32. 32 GPT-5 nano high 13156.0
  33. 33 GPT-4o Mar 2025 12622.0
  34. 34 Gemma 4 26B A4B 10667.0
  35. 35 MiMo-V2.5 9778.0
  36. 36 GPT-4.1 7022.0
  37. 37 Gemini 2.5 Flash Jun 2025 5689.0
  38. 38 sonar 3956.0
  39. 39 DeepSeek-V3 Mar 2025 3600.0
  40. 40 gpt-oss-120b 3111.0
  41. 41 Cohere Command A 2800.0
  42. 42 Kimi K2 Sep 2025 2711.0
  43. 43 GPT-4.1 mini 2489.0
  44. 44 QwQ-32B 2178.0
  45. 45 GLM-4.5-Air 1867.0
  46. 46 qwen3-235b-a22b-2507 1667.0
  47. 47 Llama 4 Maverick 1422.0
  48. 48 GPT-4o mini 1267.0
  49. 49 DeepSeek-V3.1 non-thinking 1111.0
  50. 50 nvidia-nemotron-3-nano Q4_K_M 967.0
  51. 51 Mistral Small 3.2 Q4_K_M 933.0
  52. 52 Gemini 2.5 Flash-Lite Jun 2025 700.0
  53. 53 qwen3-30b-a3b-2507 Q4_K_M 656.0
  54. 54 behemoth-123B-v1.2 611.0
  55. 55 qwen3-coder-480b 451.0
  56. 56 GPT-4.1 nano 389.0
  57. 57 Llama 3.3 70B Instruct 353.0
  58. 58 Phi-4 Q4_K_M 349.0
  59. 59 Ministral 3 14B Q4_K_M 300.0
  60. 60 hermes-4-14b Q4_K_M 296.0
  61. 61 gpt-oss-20b Q4_K_M 296.0
  62. 62 Gemma 3 27B Q4_K_M 289.0
  63. 63 glm-4.6-flash Q4_K_M 280.0
  64. 64 Gemma 3 12B Q4_K_M 244.0
  65. 65 Qwen3-4B-Thinking-2507 Q4_K_M 238.0
  66. 66 Mistral NeMo Q4_K_M 224.0
  67. 67 Ministral 3 8B Instruct Q4_K_M 211.0
  68. 68 glm-4.7-flash Q4_K_M 191.0
  69. 69 Gemma 4 12B Q4_K_M 191.0
  70. 70 Ling-flash-2.0 184.0
  71. 71 Qwen3.5-9B Q4_K_M 169.0
  72. 72 granite-3.2-8b Q4_K_M 156.0
  73. 73 apertus:8b Q4_K_M 149.0
  74. 74 Granite 4.0 H Small Q4_K_M 149.0
  75. 75 k2-horizon-7b-mlx Q4_K_M 124.0
  76. 76 Llama 3.1-8B Q4_K_M 118.0
  77. 77 Gemma 3n E4B Q4_K_M 111.0
  78. 78 qwen3-4b-instruct-2507 Q4_K_M 109.0
  79. 79 LFM2-8B-A1B Q4_K_M 88.0
  80. 80 lfm2-2.6b Q4_K_M 86.0
  81. 81 Qwen3-1.7B Q4_K_M 84.0
  82. 82 lfm2-24b-a2b Q4_K_M 80.0
  83. 83 nemotron-nano-9b-v2 Q4_K_M 79.0
  84. 84 eurollm-9b-instruct Q4_K_M 77.0
  85. 85 Qwen3.5-4B Q4_K_M 77.0
  86. 86 granite-4-h-tiny Q4_K_M 76.0
  87. 87 Gemma 4 E4B Q4_K_M 70.0
  88. 88 jamba-reasoning-3b Q4_K_M 67.0
  89. 89 Gemma 3 4B Q4_K_M 64.0
  90. 90 ling-mini-2.0 Q4_K_M 62.0
  91. 91 Phi-4 Mini Q4_K_M 58.0
  92. 92 Ministral 3 3B Instruct Q4_K_M 57.0
  93. 93 granite-4.0-h-micro Q4_K_M 53.0
  94. 94 Llama 3.2 3B Q4_K_M 50.0
  95. 95 olmo-3-7b-instruct Q4_K_M 36.0
  96. 96 lfm2.5-1.2b Q4_K_M 32.0
#EntryScoreSettingDateSource
1 GPT-6 Astra 644444.0 high Sep 2026 Official leaderboard
2 GPT-5.5 588889.0 xhigh Apr 2026 Official leaderboard
3 Gemini 3.5 Flash 433333.0 May 2026 Official leaderboard
4 GPT-5.6 Luna 416667.0 max Jul 2026 Official leaderboard
5 GPT-5 366667.0 medium Aug 2025 Official leaderboard
6 GPT-5.6 Sol 358333.0 max Jul 2026 Official leaderboard
7 GPT-5.2 333333.0 medium Dec 2025 Official leaderboard
8 GPT-5.1-Codex 291667.0 medium Nov 2025 Official leaderboard
9 GPT-5.4 275000.0 xhigh Mar 2026 Official leaderboard
10 Gemini 2.5 Pro Jun 2025 266667.0 Jun 2025 Official leaderboard
11 GPT-5.4 Mini 225000.0 Mar 2026 Official leaderboard
12 GPT-5.2 Codex 200000.0 medium Dec 2025 Official leaderboard
13 GPT-5.6 Terra 183333.0 max Jul 2026 Official leaderboard
14 DeepSeek V4.1 Flash 175000.0 Sep 2026 Official leaderboard
15 GPT-5.1 114156.0 medium Nov 2025 Official leaderboard
16 Gemini 3.1 Flash-Lite 114156.0 Mar 2026 Official leaderboard
17 Kimi K2.6 114156.0 Apr 2026 Official leaderboard
18 o3 112800.0 medium Apr 2025 Official leaderboard
19 GPT-5 mini 97889.0 high Aug 2025 Official leaderboard
20 GLM-5.2 91111.0 Jun 2026 Official leaderboard
21 Qwen3.7-Max 87044.0 May 2026 Official leaderboard
22 Grok 4.3 76200.0 high Apr 2026 Official leaderboard
23 DeepSeek-V4-Pro 73489.0 Apr 2026 Official leaderboard
24 DeepSeek-V4-Flash 61867.0 Apr 2026 Official leaderboard
25 o4-mini 56178.0 medium Apr 2025 Official leaderboard
26 GPT-5.4 Nano 27733.0 xhigh Mar 2026 Official leaderboard
27 MiniMax-M2.7 27022.0 Mar 2026 Official leaderboard
28 Gemma 4 31B IT 26311.0 Apr 2026 Official leaderboard
29 GLM-4.5 16000.0 Aug 2025 Official leaderboard
30 Qwen3-235B-A22B 15822.0 Apr 2025 Official leaderboard
31 MiniMax-M3 14756.0 Jun 2026 Official leaderboard
32 GPT-5 nano 13156.0 high Aug 2025 Official leaderboard
33 GPT-4o Mar 2025 12622.0 Mar 2025 Official leaderboard
34 Gemma 4 26B A4B 10667.0 Apr 2026 Official leaderboard
35 MiMo-V2.5 9778.0 Apr 2026 Official leaderboard
36 GPT-4.1 7022.0 Apr 2025 Official leaderboard
37 Gemini 2.5 Flash Jun 2025 5689.0 Jun 2025 Official leaderboard
38 sonar 3956.0 Jan 2025 Official leaderboard
39 DeepSeek-V3 Mar 2025 3600.0 Mar 2025 Official leaderboard
40 gpt-oss-120b 3111.0 Aug 2025 Official leaderboard
41 Cohere Command A 2800.0 Mar 2025 Official leaderboard
42 Kimi K2 Sep 2025 2711.0 Sep 2025 Official leaderboard
43 GPT-4.1 mini 2489.0 Apr 2025 Official leaderboard
44 QwQ-32B 2178.0 Mar 2025 Official leaderboard
45 GLM-4.5-Air 1867.0 Jul 2025 Official leaderboard
46 qwen3-235b-a22b-2507 1667.0 Official leaderboard
47 Llama 4 Maverick 1422.0 Apr 2025 Official leaderboard
48 GPT-4o mini 1267.0 Jul 2024 Official leaderboard
49 DeepSeek-V3.1 1111.0 non-thinking Aug 2025 Official leaderboard
50 nvidia-nemotron-3-nano 967.0 Q4_K_M Official leaderboard
51 Mistral Small 3.2 933.0 Q4_K_M Jun 2025 Official leaderboard
52 Gemini 2.5 Flash-Lite Jun 2025 700.0 Jun 2025 Official leaderboard
53 qwen3-30b-a3b-2507 656.0 Q4_K_M Official leaderboard
54 behemoth-123B-v1.2 611.0 Official leaderboard
55 qwen3-coder-480b 451.0 Official leaderboard
56 GPT-4.1 nano 389.0 Apr 2025 Official leaderboard
57 Llama 3.3 70B Instruct 353.0 Dec 2024 Official leaderboard
58 Phi-4 349.0 Q4_K_M Dec 2024 Official leaderboard
59 Ministral 3 14B 300.0 Q4_K_M Dec 2025 Official leaderboard
60 hermes-4-14b 296.0 Q4_K_M Official leaderboard
61 gpt-oss-20b 296.0 Q4_K_M Aug 2025 Official leaderboard
62 Gemma 3 27B 289.0 Q4_K_M Mar 2025 Official leaderboard
63 glm-4.6-flash 280.0 Q4_K_M Official leaderboard
64 Gemma 3 12B 244.0 Q4_K_M Mar 2025 Official leaderboard
65 Qwen3-4B-Thinking-2507 238.0 Q4_K_M Aug 2025 Official leaderboard
66 Mistral NeMo 224.0 Q4_K_M Jul 2024 Official leaderboard
67 Ministral 3 8B Instruct 211.0 Q4_K_M Dec 2025 Official leaderboard
68 glm-4.7-flash 191.0 Q4_K_M Jan 2026 Official leaderboard
69 Gemma 4 12B 191.0 Q4_K_M Jun 2026 Official leaderboard
70 Ling-flash-2.0 184.0 Sep 2025 Official leaderboard
71 Qwen3.5-9B 169.0 Q4_K_M Feb 2026 Official leaderboard
72 granite-3.2-8b 156.0 Q4_K_M Official leaderboard
73 apertus:8b 149.0 Q4_K_M Official leaderboard
74 Granite 4.0 H Small 149.0 Q4_K_M Oct 2025 Official leaderboard
75 k2-horizon-7b-mlx 124.0 Q4_K_M Official leaderboard
76 Llama 3.1-8B 118.0 Q4_K_M Jul 2024 Official leaderboard
77 Gemma 3n E4B 111.0 Q4_K_M Jun 2025 Official leaderboard
78 qwen3-4b-instruct-2507 109.0 Q4_K_M Aug 2025 Official leaderboard
79 LFM2-8B-A1B 88.0 Q4_K_M Oct 2025 Official leaderboard
80 lfm2-2.6b 86.0 Q4_K_M Official leaderboard
81 Qwen3-1.7B 84.0 Q4_K_M Apr 2025 Official leaderboard
82 lfm2-24b-a2b 80.0 Q4_K_M Official leaderboard
83 nemotron-nano-9b-v2 79.0 Q4_K_M Official leaderboard
84 eurollm-9b-instruct 77.0 Q4_K_M Official leaderboard
85 Qwen3.5-4B 77.0 Q4_K_M Feb 2026 Official leaderboard
86 granite-4-h-tiny 76.0 Q4_K_M Official leaderboard
87 Gemma 4 E4B 70.0 Q4_K_M Mar 2026 Official leaderboard
88 jamba-reasoning-3b 67.0 Q4_K_M Official leaderboard
89 Gemma 3 4B 64.0 Q4_K_M Mar 2025 Official leaderboard
90 ling-mini-2.0 62.0 Q4_K_M Official leaderboard
91 Phi-4 Mini 58.0 Q4_K_M Feb 2025 Official leaderboard
92 Ministral 3 3B Instruct 57.0 Q4_K_M Dec 2025 Official leaderboard
93 granite-4.0-h-micro 53.0 Q4_K_M Official leaderboard
94 Llama 3.2 3B 50.0 Q4_K_M Sep 2024 Official leaderboard
95 olmo-3-7b-instruct 36.0 Q4_K_M Official leaderboard
96 lfm2.5-1.2b 32.0 Q4_K_M Official leaderboard

3 models have held number one since its release in Sep 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Sep 2026
GPT-6 Astra 644444.0

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 200000.0 400000.0 600000.0 800000.0 2025 2026 0.00 200000.0 400000.0 600000.0 800000.0 2025 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Multilingual

All 115 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Inkling 88.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%