Benchmark Almanac

BullshitBench V2

Claude Opus 4.8 95% leads since May 2026 Not in model cards

Models answer 100 nonsense questions across software, finance, legal, medical and physics, built with 13 techniques. A three-judge panel scores clear pushback.

Released 2026-03-01 with a new question set, so scores do not compare with V1. 236 model variants by 2026-10-13.

Made by Peter Gostev Released Mar 2026 Hallucination rates All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
petergpt/bullshit-benchmark 1,903 stars Leaderboard petergpt.github.io
130
Entries on its own leaderboard, 122 models we track

2 models share the top score of 95%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

130 entries. Scroll sideways for the rest.

  1. 95.0 Claude Opus 4.8
  2. 95.0 Qwen 3.8 Max
  3. 91.0 Claude Sonnet 4.6
  4. 90.0 Claude Opus 4.5
  5. 87.0 Claude Opus 4.6
  6. 86.5 Claude Fable 5.1
  7. 83.6 Claude Fable 5
  8. 83.0 Claude Opus 4.7
  9. 80.8 Claude Sonnet 5
  10. 80.2 Claude Opus 5
  11. 80.0 Mistral Large 4
  12. 79.0 Claude Sonnet 4.5
  13. 79.0 Qwen 3.8 27B
  14. 78.0 Qwen3.5 397B-A17B
  15. 77.0 Claude Haiku 4.5
  16. 74.0 Kimi K3
  17. 72.0 Qwen 3.6 Plus
  18. 72.0 Qwen3.7-Max
  19. 72.0 GLM-5.3
  20. 69.2 Claude Opus 5.5
  21. 69.0 GPT-6 Astra
  22. 67.0 Grok 4.20 Multi-agent
  23. 66.0 Gemini 3.5 Flash-Lite
  24. 66.0 Grok 4.6
  25. 66.0 Gemini 4 Argon
  26. 65.0 stealth/ox-alpha
  27. 65.0 Kimi K2.6
  28. 65.0 GPT-6.1 Sol
  29. 64.0 MiniMax-M3
  30. 63.0 Qwen3-Max
  31. 62.0 MiMo-V2.5-Pro
  32. 60.0 Claude Sonnet 5.5
  33. 59.0 openai/gpt-6-luna-pro
  34. 59.0 GPT-6 Luna
  35. 59.0 GPT-6 Sol
  36. 57.6 Claude Haiku 5.5
  37. 57.6 Muse Wonder
  38. 57.0 openai/gpt-6-sol-pro
  39. 57.0 DeepSeek V4.1 Flash
  40. 56.0 Grok 4.20
  41. 55.0 Grok 4.5
  42. 54.0 Nemotron 3 Super
  43. 54.0 GPT-5.6 Terra
  44. 52.0 Kimi K2.5
  45. 50.0 Claude 3.5 Haiku
  46. 50.0 Grok 4.3
  47. 50.0 Muse Spark 1.2
  48. 49.0 Claude 3.7 Sonnet
  49. 49.0 Nemotron 3 Ultra
  50. 48.0 Gemini 3 Pro
  51. 48.0 GPT-5.4
  52. 48.0 GPT-5.6 Sol
  53. 47.0 GPT-5.5
  54. 46.0 Claude 3.5 Sonnet
  55. 45.0 GPT-5.1 Instant
  56. 45.0 GPT-5.2 Codex
  57. 43.0 openrouter/hunter-alpha
  58. 43.0 Claude Opus 4.1
  59. 41.0 GPT-5.6 Luna
  60. 40.0 GPT-5.3 Instant
  61. 39.0 GPT-5-Codex
  62. 39.0 Gemini 3.6 Flash
  63. 39.0 DeepSeek V4 Flash 0731
  64. 38.0 GPT-5.2
  65. 37.0 openrouter/healer-alpha
  66. 37.0 Gemini 3.1 Pro
  67. 36.0 GPT-5.5 Pro
  68. 35.0 MiMo-V2.5
  69. 35.0 DeepSeek V4 Pro 0813
  70. 35.0 Gemini 3.7 Flash
  71. 34.0 openai/gpt-5.5-chat
  72. 34.0 Claude Opus 4
  73. 32.0 GPT-5.1-Codex
  74. 32.0 GPT-5.4 Mini
  75. 31.0 GLM-5-Turbo
  76. 31.0 GLM-5.2
  77. 30.0 Claude Sonnet 4
  78. 29.0 Llama 4 Maverick
  79. 28.0 Nemotron 3 Nano 30B-A3B
  80. 28.0 GLM-5
  81. 27.0 GPT-5.2 Instant
  82. 26.0 o3
  83. 25.0 GPT-5.1
  84. 25.0 Gemma 4 31B IT
  85. 24.0 GPT-5.3 Codex
  86. 22.0 Step 3.5 Flash
  87. 22.0 GLM-5.1
  88. 22.0 Laguna S 2.1
  89. 21.0 GPT-5
  90. 21.0 Gemma 4 26B A4B
  91. 20.0 Llama 4 Scout
  92. 20.0 Gemini 2.5 Pro
  93. 20.0 Qwen3-Coder-480B-A35B
  94. 20.0 Gemini 3.5 Flash
  95. 19.0 Gemini 2.5 Flash
  96. 19.0 Grok 4.1 Fast
  97. 18.0 GPT-5 Chat
  98. 18.0 DeepSeek-V4-Flash
  99. 17.0 Trinity Large Thinking
  100. 16.0 MiMo-V2-Flash
  101. 16.0 Tencent Hy3 preview
  102. 15.0 Gemini 2.0 Flash
  103. 14.0 Llama 3.1-8B
  104. 14.0 GPT-4.1
  105. 14.0 GPT-5.4 Nano
  106. 14.0 DeepSeek-V4-Pro
  107. 13.0 DeepSeek-V3.2
  108. 12.0 GPT-4o
  109. 12.0 gpt-oss-120b
  110. 11.0 baidu/ernie-4.5-vl-424b-a47b
  111. 11.0 Doubao-Seed-1.6
  112. 11.0 Gemini 3.1 Flash-Lite
  113. 10.0 Claude 3 Haiku
  114. 10.0 Kimi K2
  115. 10.0 Gemini 3 Flash
  116. 9.00 MiniMax-M2.5
  117. 8.00 o4-mini
  118. 8.00 DeepSeek-R1
  119. 8.00 GLM-4.5
  120. 8.00 MiniMax-M2.7
  121. 7.00 INTELLECT-3
  122. 6.00 Qwen3-235B-A22B
  123. 6.00 Mistral Small 4
  124. 5.00 Jamba 1.7 Large
  125. 5.00 nvidia-nemotron-nano-9b-v2
  126. 4.00 deepseek/deepseek-chat
  127. 4.00 ERNIE-4.5-300B-A47B
  128. 3.00 Gemma 3 27B
  129. 2.00 GPT-4o mini
  130. 2.00 Mistral Large 3
  1. 1 Claude Opus 4.8 none 95%
  2. 2 Qwen 3.8 Max minimal 95%
  3. 3 Claude Sonnet 4.6 high 91%
  4. 4 Claude Opus 4.5 high 90%
  5. 5 Claude Opus 4.6 high 87%
  6. 6 Claude Fable 5.1 low 86.5%
  7. 7 Claude Fable 5 xhigh 83.6%
  8. 8 Claude Opus 4.7 none 83%
  9. 9 Claude Sonnet 5 low 80.8%
  10. 10 Claude Opus 5 low 80.2%
  11. 11 Mistral Large 4 none 80%
  12. 12 Claude Sonnet 4.5 high 79%
  13. 13 Qwen 3.8 27B none 79%
  14. 14 Qwen3.5 397B-A17B high 78%
  15. 15 Claude Haiku 4.5 high 77%
  16. 16 Kimi K3 xhigh 74%
  17. 17 Qwen 3.6 Plus none 72%
  18. 18 Qwen3.7-Max none 72%
  19. 19 GLM-5.3 minimal 72%
  20. 20 Claude Opus 5.5 low 69.2%
  21. 21 GPT-6 Astra max 69%
  22. 22 Grok 4.20 Multi-agent low 67%
  23. 23 Gemini 3.5 Flash-Lite minimal 66%
  24. 24 Grok 4.6 minimal 66%
  25. 25 Gemini 4 Argon low 66%
  26. 26 stealth/ox-alpha low 65%
  27. 27 Kimi K2.6 none 65%
  28. 28 GPT-6.1 Sol low 65%
  29. 29 MiniMax-M3 xhigh 64%
  30. 30 Qwen3-Max high 63%
  31. 31 MiMo-V2.5-Pro xhigh 62%
  32. 32 Claude Sonnet 5.5 low 60%
  33. 33 openai/gpt-6-luna-pro max 59%
  34. 34 GPT-6 Luna none 59%
  35. 35 GPT-6 Sol none 59%
  36. 36 Claude Haiku 5.5 max 57.6%
  37. 37 Muse Wonder minimal 57.6%
  38. 38 openai/gpt-6-sol-pro max 57%
  39. 39 DeepSeek V4.1 Flash max 57%
  40. 40 Grok 4.20 low 56%
  41. 41 Grok 4.5 high 55%
  42. 42 Nemotron 3 Super xhigh 54%
  43. 43 GPT-5.6 Terra max 54%
  44. 44 Kimi K2.5 none 52%
  45. 45 Claude 3.5 Haiku 50%
  46. 46 Grok 4.3 minimal 50%
  47. 47 Muse Spark 1.2 minimal 50%
  48. 48 Claude 3.7 Sonnet thinking 49%
  49. 49 Nemotron 3 Ultra none 49%
  50. 50 Gemini 3 Pro low 48%
  51. 51 GPT-5.4 none 48%
  52. 52 GPT-5.6 Sol low 48%
  53. 53 GPT-5.5 xhigh 47%
  54. 54 Claude 3.5 Sonnet 46%
  55. 55 GPT-5.1 Instant 45%
  56. 56 GPT-5.2 Codex low 45%
  57. 57 openrouter/hunter-alpha none 43%
  58. 58 Claude Opus 4.1 none 43%
  59. 59 GPT-5.6 Luna max 41%
  60. 60 GPT-5.3 Instant 40%
  61. 61 GPT-5-Codex 39%
  62. 62 Gemini 3.6 Flash minimal 39%
  63. 63 DeepSeek V4 Flash 0731 xhigh 39%
  64. 64 GPT-5.2 none 38%
  65. 65 openrouter/healer-alpha none 37%
  66. 66 Gemini 3.1 Pro low 37%
  67. 67 GPT-5.5 Pro xhigh 36%
  68. 68 MiMo-V2.5 xhigh 35%
  69. 69 DeepSeek V4 Pro 0813 max 35%
  70. 70 Gemini 3.7 Flash minimal 35%
  71. 71 openai/gpt-5.5-chat 34%
  72. 72 Claude Opus 4 34%
  73. 73 GPT-5.1-Codex 32%
  74. 74 GPT-5.4 Mini high 32%
  75. 75 GLM-5-Turbo high 31%
  76. 76 GLM-5.2 xhigh 31%
  77. 77 Claude Sonnet 4 high 30%
  78. 78 Llama 4 Maverick 29%
  79. 79 Nemotron 3 Nano 30B-A3B none 28%
  80. 80 GLM-5 high 28%
  81. 81 GPT-5.2 Instant 27%
  82. 82 o3 26%
  83. 83 GPT-5.1 25%
  84. 84 Gemma 4 31B IT high 25%
  85. 85 GPT-5.3 Codex low 24%
  86. 86 Step 3.5 Flash xhigh 22%
  87. 87 GLM-5.1 xhigh 22%
  88. 88 Laguna S 2.1 none 22%
  89. 89 GPT-5 21%
  90. 90 Gemma 4 26B A4B xhigh 21%
  91. 91 Llama 4 Scout 20%
  92. 92 Gemini 2.5 Pro Jun 2025 20%
  93. 93 Qwen3-Coder-480B-A35B 20%
  94. 94 Gemini 3.5 Flash xhigh 20%
  95. 95 Gemini 2.5 Flash Sep 2025 19%
  96. 96 Grok 4.1 Fast high 19%
  97. 97 GPT-5 Chat 18%
  98. 98 DeepSeek-V4-Flash none 18%
  99. 99 Trinity Large Thinking minimal 17%
  100. 100 MiMo-V2-Flash none 16%
  101. 101 Tencent Hy3 preview none 16%
  102. 102 Gemini 2.0 Flash Feb 2025 15%
  103. 103 Llama 3.1-8B 14%
  104. 104 GPT-4.1 14%
  105. 105 GPT-5.4 Nano high 14%
  106. 106 DeepSeek-V4-Pro xhigh 14%
  107. 107 DeepSeek-V3.2 high 13%
  108. 108 GPT-4o Aug 2024 12%
  109. 109 gpt-oss-120b low 12%
  110. 110 baidu/ernie-4.5-vl-424b-a47b xhigh 11%
  111. 111 Doubao-Seed-1.6 none 11%
  112. 112 Gemini 3.1 Flash-Lite 11%
  113. 113 Claude 3 Haiku 10%
  114. 114 Kimi K2 Sep 2025 10%
  115. 115 Gemini 3 Flash high 10%
  116. 116 MiniMax-M2.5 low 9.00%
  117. 117 o4-mini low 8.00%
  118. 118 DeepSeek-R1 May 2025, xhigh 8.00%
  119. 119 GLM-4.5 xhigh 8.00%
  120. 120 MiniMax-M2.7 high 8.00%
  121. 121 INTELLECT-3 low 7.00%
  122. 122 Qwen3-235B-A22B none 6.00%
  123. 123 Mistral Small 4 high 6.00%
  124. 124 Jamba 1.7 Large 5.00%
  125. 125 nvidia-nemotron-nano-9b-v2 none 5.00%
  126. 126 deepseek/deepseek-chat 4.00%
  127. 127 ERNIE-4.5-300B-A47B 4.00%
  128. 128 Gemma 3 27B 3.00%
  129. 129 GPT-4o mini 2.00%
  130. 130 Mistral Large 3 2.00%
#EntryScoreSettingDateSource
1 Claude Opus 4.8 95% none May 2026 Official leaderboard
2 Qwen 3.8 Max 95% minimal Aug 2026 Official leaderboard
3 Claude Sonnet 4.6 91% high Feb 2026 Official leaderboard
4 Claude Opus 4.5 90% high Nov 2025 Official leaderboard
5 Claude Opus 4.6 87% high Feb 2026 Official leaderboard
6 Claude Fable 5.1 86.5% low Sep 2026 Official leaderboard
7 Claude Fable 5 83.6% xhigh Jun 2026 Official leaderboard
8 Claude Opus 4.7 83% none Apr 2026 Official leaderboard
9 Claude Sonnet 5 80.8% low Jun 2026 Official leaderboard
10 Claude Opus 5 80.2% low Jul 2026 Official leaderboard
11 Mistral Large 4 80% none Oct 2026 Official leaderboard
12 Claude Sonnet 4.5 79% high Sep 2025 Official leaderboard
13 Qwen 3.8 27B 79% none Aug 2026 Official leaderboard
14 Qwen3.5 397B-A17B 78% high Feb 2026 Official leaderboard
15 Claude Haiku 4.5 77% high Oct 2025 Official leaderboard
16 Kimi K3 74% xhigh Jul 2026 Official leaderboard
17 Qwen 3.6 Plus 72% none Mar 2026 Official leaderboard
18 Qwen3.7-Max 72% none May 2026 Official leaderboard
19 GLM-5.3 72% minimal Aug 2026 Official leaderboard
20 Claude Opus 5.5 69.2% low Sep 2026 Official leaderboard
21 GPT-6 Astra 69% max Sep 2026 Official leaderboard
22 Grok 4.20 Multi-agent 67% low Mar 2026 Official leaderboard
23 Gemini 3.5 Flash-Lite 66% minimal Jul 2026 Official leaderboard
24 Grok 4.6 66% minimal Aug 2026 Official leaderboard
25 Gemini 4 Argon 66% low Sep 2026 Official leaderboard
26 stealth/ox-alpha 65% low Official leaderboard
27 Kimi K2.6 65% none Apr 2026 Official leaderboard
28 GPT-6.1 Sol 65% low Sep 2026 Official leaderboard
29 MiniMax-M3 64% xhigh Jun 2026 Official leaderboard
30 Qwen3-Max 63% high Sep 2025 Official leaderboard
31 MiMo-V2.5-Pro 62% xhigh Apr 2026 Official leaderboard
32 Claude Sonnet 5.5 60% low Sep 2026 Official leaderboard
33 openai/gpt-6-luna-pro 59% max Official leaderboard
34 GPT-6 Luna 59% none Sep 2026 Official leaderboard
35 GPT-6 Sol 59% none Sep 2026 Official leaderboard
36 Claude Haiku 5.5 57.6% max Oct 2026 Official leaderboard
37 Muse Wonder 57.6% minimal Oct 2026 Official leaderboard
38 openai/gpt-6-sol-pro 57% max Official leaderboard
39 DeepSeek V4.1 Flash 57% max Sep 2026 Official leaderboard
40 Grok 4.20 56% low Feb 2026 Official leaderboard
41 Grok 4.5 55% high Jul 2026 Official leaderboard
42 Nemotron 3 Super 54% xhigh Mar 2026 Official leaderboard
43 GPT-5.6 Terra 54% max Jul 2026 Official leaderboard
44 Kimi K2.5 52% none Jan 2026 Official leaderboard
45 Claude 3.5 Haiku 50% Oct 2024 Official leaderboard
46 Grok 4.3 50% minimal Apr 2026 Official leaderboard
47 Muse Spark 1.2 50% minimal Aug 2026 Official leaderboard
48 Claude 3.7 Sonnet 49% thinking Feb 2025 Official leaderboard
49 Nemotron 3 Ultra 49% none Jun 2026 Official leaderboard
50 Gemini 3 Pro 48% low Nov 2025 Official leaderboard
51 GPT-5.4 48% none Mar 2026 Official leaderboard
52 GPT-5.6 Sol 48% low Jul 2026 Official leaderboard
53 GPT-5.5 47% xhigh Apr 2026 Official leaderboard
54 Claude 3.5 Sonnet 46% Jun 2024 Official leaderboard
55 GPT-5.1 Instant 45% Nov 2025 Official leaderboard
56 GPT-5.2 Codex 45% low Dec 2025 Official leaderboard
57 openrouter/hunter-alpha 43% none Official leaderboard
58 Claude Opus 4.1 43% none Aug 2025 Official leaderboard
59 GPT-5.6 Luna 41% max Jul 2026 Official leaderboard
60 GPT-5.3 Instant 40% Mar 2026 Official leaderboard
61 GPT-5-Codex 39% Sep 2025 Official leaderboard
62 Gemini 3.6 Flash 39% minimal Jul 2026 Official leaderboard
63 DeepSeek V4 Flash 0731 39% xhigh Jul 2026 Official leaderboard
64 GPT-5.2 38% none Dec 2025 Official leaderboard
65 openrouter/healer-alpha 37% none Official leaderboard
66 Gemini 3.1 Pro 37% low Feb 2026 Official leaderboard
67 GPT-5.5 Pro 36% xhigh Apr 2026 Official leaderboard
68 MiMo-V2.5 35% xhigh Apr 2026 Official leaderboard
69 DeepSeek V4 Pro 0813 35% max Aug 2026 Official leaderboard
70 Gemini 3.7 Flash 35% minimal Aug 2026 Official leaderboard
71 openai/gpt-5.5-chat 34% Official leaderboard
72 Claude Opus 4 34% May 2025 Official leaderboard
73 GPT-5.1-Codex 32% Nov 2025 Official leaderboard
74 GPT-5.4 Mini 32% high Mar 2026 Official leaderboard
75 GLM-5-Turbo 31% high Mar 2026 Official leaderboard
76 GLM-5.2 31% xhigh Jun 2026 Official leaderboard
77 Claude Sonnet 4 30% high May 2025 Official leaderboard
78 Llama 4 Maverick 29% Apr 2025 Official leaderboard
79 Nemotron 3 Nano 30B-A3B 28% none Dec 2025 Official leaderboard
80 GLM-5 28% high Feb 2026 Official leaderboard
81 GPT-5.2 Instant 27% Dec 2025 Official leaderboard
82 o3 26% Apr 2025 Official leaderboard
83 GPT-5.1 25% Nov 2025 Official leaderboard
84 Gemma 4 31B IT 25% high Apr 2026 Official leaderboard
85 GPT-5.3 Codex 24% low Feb 2026 Official leaderboard
86 Step 3.5 Flash 22% xhigh Feb 2026 Official leaderboard
87 GLM-5.1 22% xhigh Apr 2026 Official leaderboard
88 Laguna S 2.1 22% none Jul 2026 Official leaderboard
89 GPT-5 21% Aug 2025 Official leaderboard
90 Gemma 4 26B A4B 21% xhigh Apr 2026 Official leaderboard
91 Llama 4 Scout 20% Apr 2025 Official leaderboard
92 Gemini 2.5 Pro Jun 2025 20% Jun 2025 Official leaderboard
93 Qwen3-Coder-480B-A35B 20% Jul 2025 Official leaderboard
94 Gemini 3.5 Flash 20% xhigh May 2026 Official leaderboard
95 Gemini 2.5 Flash Sep 2025 19% Sep 2025 Official leaderboard
96 Grok 4.1 Fast 19% high Nov 2025 Official leaderboard
97 GPT-5 Chat 18% Aug 2025 Official leaderboard
98 DeepSeek-V4-Flash 18% none Apr 2026 Official leaderboard
99 Trinity Large Thinking 17% minimal Apr 2026 Official leaderboard
100 MiMo-V2-Flash 16% none Dec 2025 Official leaderboard
101 Tencent Hy3 preview 16% none Apr 2026 Official leaderboard
102 Gemini 2.0 Flash Feb 2025 15% Feb 2025 Official leaderboard
103 Llama 3.1-8B 14% Jul 2024 Official leaderboard
104 GPT-4.1 14% Apr 2025 Official leaderboard
105 GPT-5.4 Nano 14% high Mar 2026 Official leaderboard
106 DeepSeek-V4-Pro 14% xhigh Apr 2026 Official leaderboard
107 DeepSeek-V3.2 13% high Dec 2025 Official leaderboard
108 GPT-4o Aug 2024 12% Aug 2024 Official leaderboard
109 gpt-oss-120b 12% low Aug 2025 Official leaderboard
110 baidu/ernie-4.5-vl-424b-a47b 11% xhigh Official leaderboard
111 Doubao-Seed-1.6 11% none Jun 2025 Official leaderboard
112 Gemini 3.1 Flash-Lite 11% Mar 2026 Official leaderboard
113 Claude 3 Haiku 10% Mar 2024 Official leaderboard
114 Kimi K2 Sep 2025 10% Sep 2025 Official leaderboard
115 Gemini 3 Flash 10% high Dec 2025 Official leaderboard
116 MiniMax-M2.5 9.00% low Feb 2026 Official leaderboard
117 o4-mini 8.00% low Apr 2025 Official leaderboard
118 DeepSeek-R1 May 2025 8.00% xhigh May 2025 Official leaderboard
119 GLM-4.5 8.00% xhigh Aug 2025 Official leaderboard
120 MiniMax-M2.7 8.00% high Mar 2026 Official leaderboard
121 INTELLECT-3 7.00% low Nov 2025 Official leaderboard
122 Qwen3-235B-A22B 6.00% none Apr 2025 Official leaderboard
123 Mistral Small 4 6.00% high Mar 2026 Official leaderboard
124 Jamba 1.7 Large 5.00% Jul 2025 Official leaderboard
125 nvidia-nemotron-nano-9b-v2 5.00% none Aug 2025 Official leaderboard
126 deepseek/deepseek-chat 4.00% Official leaderboard
127 ERNIE-4.5-300B-A47B 4.00% Jun 2025 Official leaderboard
128 Gemma 3 27B 3.00% Mar 2025 Official leaderboard
129 GPT-4o mini 2.00% Jul 2024 Official leaderboard
130 Mistral Large 3 2.00% Dec 2025 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.8
none
95%
Qwen 3.8 Max
minimal max
95%
Claude Sonnet 4.6
none
91%
Claude Opus 4.5
none high
90%
Claude Opus 4.6
none
87%
Claude Fable 5.1
low max
86.5%
Claude Fable 5
low xhigh
83.6%
Claude Opus 4.7
none max
83%
Claude Sonnet 5
low
80.8%
Claude Opus 5
low
80.2%
Mistral Large 4
none high
80%
Claude Sonnet 4.5
none
79%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 none 95%
Qwen 3.8 Max minimal 95%
Claude Opus 4.8 xhigh 94%
Claude Sonnet 4.6 high 91%
Claude Opus 4.5 high 90%
Claude Sonnet 4.6 none 89%
Claude Opus 4.6 high 87%
Claude Fable 5.1 low 86.5%
Claude Fable 5 xhigh 83.6%
Claude Opus 4.6 none 83%
Claude Opus 4.7 none 83%
Claude Sonnet 5 low 80.8%
Claude Opus 5 low 80.2%
Mistral Large 4 none 80%
Claude Sonnet 4.5 high 79%
Claude Opus 4.5 none 79%
Qwen 3.8 27B none 79%
Claude Sonnet 5 max 78.8%
Claude Opus 5 xhigh 78.7%
Qwen3.5 397B-A17B high 78%
Claude Haiku 4.5 high 77%
Claude Fable 5.1 max 74.4%
Claude Sonnet 4.5 none 74%
Claude Opus 4.7 max 74%
Kimi K3 xhigh 74%
Qwen 3.6 Plus none 72%
Qwen3.7-Max none 72%
Kimi K3 minimal 72%
GLM-5.3 minimal 72%
Claude Haiku 4.5 none 71%
Claude Fable 5 low 70.3%
Claude Opus 5.5 low 69.2%
Qwen3.5 397B-A17B none 69%
GPT-6 Astra max 69%
Claude Opus 5.5 max 68.9%
Mistral Large 4 high 68%
Grok 4.20 Multi-agent low 67%
Gemini 3.5 Flash-Lite minimal 66%
Grok 4.6 minimal 66%
GLM-5.3 max 66%
Gemini 4 Argon low 66%
Kimi K2.6 none 65%
GPT-6.1 Sol low 65%
Gemini 4 Argon high 65%
Grok 4.20 Multi-agent xhigh 64%
MiniMax-M3 xhigh 64%
GPT-6 Astra low 64%
Qwen3-Max high 63%
MiniMax-M3 none 63%
MiMo-V2.5-Pro xhigh 62%
Gemini 3.5 Flash-Lite xhigh 60%
Claude Sonnet 5.5 low 60%
Qwen 3.6 Plus xhigh 59%
GPT-6 Luna none 59%
GPT-6 Sol none 59%
GPT-6.1 Sol max 59%
GPT-6 Luna max 58%
Claude Sonnet 5.5 max 58%
Claude Haiku 5.5 max 57.6%
Muse Wonder minimal 57.6%
Qwen3.7-Max xhigh 57%
DeepSeek V4.1 Flash max 57%
DeepSeek V4.1 Flash low 57%
Grok 4.20 low 56%
Grok 4.6 max 56%
Grok 4.5 high 55%
Grok 4.20 xhigh 54%
Nemotron 3 Super xhigh 54%
Grok 4.5 low 54%
GPT-5.6 Terra max 54%
GPT-6 Sol max 54%
Claude Haiku 5.5 low 53%
Muse Wonder max 52.5%
Kimi K2.5 none 52%
Claude 3.5 Haiku 50%
Kimi K2.6 xhigh 50%
Grok 4.3 minimal 50%
Muse Spark 1.2 minimal 50%
Claude 3.7 Sonnet thinking 49%
Nemotron 3 Ultra none 49%
Muse Spark 1.2 xhigh 49%
Gemini 3 Pro low 48%
GPT-5.4 none 48%
GPT-5.6 Sol low 48%
Nemotron 3 Super high 47%
GPT-5.5 xhigh 47%
GPT-5.6 Sol max 47%
GPT-5.6 Terra low 47%
Claude 3.5 Sonnet 46%
Qwen3-Max none 46%
Grok 4.3 xhigh 46%
GPT-5.1 Instant 45%
GPT-5.2 Codex low 45%
GPT-5.5 none 45%
GPT-5.5 low 45%
Claude 3.7 Sonnet 44%
Claude Opus 4.1 none 43%
Nemotron 3 Super none 43%
Claude Opus 4.1 high 42%
GPT-5.4 xhigh 42%
GPT-5.6 Luna max 41%
GPT-5.3 Instant 40%
Nemotron 3 Ultra xhigh 40%
GPT-5-Codex 39%
GPT-5.2 Codex xhigh 39%
Gemini 3.6 Flash minimal 39%
DeepSeek V4 Flash 0731 xhigh 39%
GPT-5.2 none 38%
MiMo-V2.5-Pro none 38%
GPT-5.2 Codex high 37%
Gemini 3.1 Pro low 37%
GPT-5.6 Luna low 37%
Gemini 3 Pro high 36%
GPT-5.5 Pro xhigh 36%
MiMo-V2.5 xhigh 35%
DeepSeek V4 Pro 0813 max 35%
Gemini 3.7 Flash minimal 35%
Claude Opus 4 34%
GPT-5.5 Pro medium 34%
GPT-5.1-Codex 32%
GPT-5.4 Mini high 32%
GPT-5.4 Mini none 32%
DeepSeek V4 Pro 0813 low 32%
Kimi K2.5 high 31%
Gemini 3.1 Pro high 31%
Nemotron 3 Super none 31%
GLM-5-Turbo high 31%
GPT-5.4 Mini xhigh 31%
GLM-5.2 xhigh 31%
Gemini 3.7 Flash max 31%
Claude Sonnet 4 high 30%
Llama 4 Maverick 29%
Claude Sonnet 4 none 29%
Qwen 3.8 27B max 29%
GPT-5.2 high 28%
Nemotron 3 Nano 30B-A3B none 28%
GLM-5 high 28%
Gemini 3.6 Flash xhigh 28%
GPT-5.2 Instant 27%
Qwen 3.8 Max max 27%
o3 26%
GPT-5.1 25%
Gemma 4 31B IT high 25%
GPT-5.3 Codex low 24%
MiMo-V2.5 none 24%
GLM-5-Turbo none 23%
Step 3.5 Flash xhigh 22%
GLM-5.1 xhigh 22%
Laguna S 2.1 none 22%
GPT-5 21%
Gemma 4 26B A4B xhigh 21%
Laguna S 2.1 xhigh 21%
Llama 4 Scout 20%
Gemini 2.5 Pro Jun 2025 20%
Qwen3-Coder-480B-A35B 20%
GPT-5.3 Codex high 20%
GLM-5 none 20%
Gemma 4 31B IT none 20%
Gemini 3.5 Flash xhigh 20%
DeepSeek V4 Flash 0731 none 20%
Gemini 2.5 Flash Sep 2025 19%
Grok 4.1 Fast high 19%
GPT-5.3 Codex xhigh 19%
Gemini 3.5 Flash minimal 19%
GPT-5 Chat 18%
GLM-5.1 none 18%
DeepSeek-V4-Flash none 18%
Trinity Large Thinking minimal 17%
GLM-5.2 none 17%
MiMo-V2-Flash none 16%
Tencent Hy3 preview none 16%
Gemini 2.0 Flash Feb 2025 15%
Llama 3.1-8B 14%
GPT-4.1 14%
GPT-5.4 Nano high 14%
GPT-5.4 Nano none 14%
DeepSeek-V4-Flash xhigh 14%
DeepSeek-V4-Pro xhigh 14%
DeepSeek-V4-Pro none 14%
DeepSeek-V3.2 high 13%
MiMo-V2-Flash high 13%
Step 3.5 Flash minimal 13%
Trinity Large Thinking xhigh 13%
GPT-4o Aug 2024 12%
gpt-oss-120b low 12%
Doubao-Seed-1.6 none 11%
Gemini 3.1 Flash-Lite 11%
Gemma 4 26B A4B none 11%
Claude 3 Haiku 10%
Kimi K2 Sep 2025 10%
Grok 4.1 Fast none 10%
DeepSeek-V3.2 none 10%
Nemotron 3 Nano 30B-A3B xhigh 10%
Gemini 3 Flash high 10%
Gemini 3 Flash none 10%
GPT-5.4 Nano xhigh 10%
MiniMax-M2.5 low 9.00%
o4-mini low 8.00%
DeepSeek-R1 May 2025 xhigh 8.00%
GLM-4.5 xhigh 8.00%
MiniMax-M2.5 high 8.00%
MiniMax-M2.7 high 8.00%
Tencent Hy3 preview xhigh 8.00%
DeepSeek-R1 May 2025 none 7.00%
Doubao-Seed-1.6 high 7.00%
INTELLECT-3 low 7.00%
MiniMax-M2.7 low 7.00%
Qwen3-235B-A22B none 6.00%
GLM-4.5 none 6.00%
Mistral Small 4 high 6.00%
Jamba 1.7 Large 5.00%
gpt-oss-120b high 5.00%
nvidia-nemotron-nano-9b-v2 none 5.00%
INTELLECT-3 high 5.00%
o4-mini high 4.00%
ERNIE-4.5-300B-A47B 4.00%
Mistral Small 4 none 4.00%
Gemma 3 27B 3.00%
Qwen3-235B-A22B xhigh 3.00%
nvidia-nemotron-nano-9b-v2 xhigh 3.00%
GPT-4o mini 2.00%
Mistral Large 3 2.00%

2 models have held number one since its release in Mar 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%

The best score passed 85% in Nov 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Opus 4.5 Claude Sonnet 4.5 Qwen3-Max Claude 3.5 Haiku Claude 3 Haiku 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Sonnet 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%
BullshitBench Passed 85% in Feb 2026, by Claude Opus 4.5
Claude Opus 4.8 96.4%
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%
Replaces BullshitBench

More in Factuality

All 70 in Factuality
  • SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. Qwen3.7-Plus 81.7%
  • HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy. Keye-VL-2.0-30B-A3B 73.5%
  • SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. Gemini 3.1 Pro 77.5%
  • GaRAGe In use Models write long-form RAG answers to 2,366 questions with human-annotated grounding passages. Scoring checks whether answers use the relevant passages. Nemotron 3.5 Bolt E4B 40.7%
  • Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. Kimi K3.1 Pro 8.30%
  • RAGTruth In use A word-level annotated corpus of hallucinations in retrieval-augmented generation answers. It is used to train and evaluate hallucination detection and RAG faithfulness. Nemotron 3.5 Bolt E4B 77.9%