Benchmark Almanac

BullshitBench

Claude Opus 4.8 96.4% leads since May 2026 Not in model cards

Models get prompts built on nonsensical premises. A three-judge model panel grades whether each answer pushes back clearly, challenges partly or accepts the nonsense.

Repo has about 1,900 GitHub stars. The V1 question set stays unchanged after V2.

Made by Peter Gostev Released Feb 2026 Hallucination rates All tasks public Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
petergpt/bullshit-benchmark 1,903 stars Leaderboard petergpt.github.io
115
Entries on its own leaderboard, 107 models we track

Claude Opus 4.8 leads with 96.4%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

115 entries. Scroll sideways for the rest.

  1. 96.4 Claude Opus 4.8
  2. 94.5 Claude Sonnet 4.6
  3. 92.7 Claude Opus 4.6
  4. 90.9 Claude Opus 4.5
  5. 87.3 Claude Haiku 4.5
  6. 87.3 Qwen 3.8 Max
  7. 80.0 Claude Opus 4.7
  8. 76.4 Qwen 3.8 27B
  9. 71.7 Claude Fable 5.1
  10. 67.3 Grok 4.20 Multi-agent
  11. 65.5 Claude Sonnet 4.5
  12. 65.5 Qwen3.5 397B-A17B
  13. 65.5 Nemotron 3 Super
  14. 65.5 Claude Sonnet 5
  15. 63.6 Grok 4.6
  16. 61.8 Grok 4.20
  17. 61.8 Grok 4.3
  18. 60.0 Grok 4.5
  19. 60.0 Claude Opus 5
  20. 58.2 Claude 3.5 Haiku
  21. 58.2 Kimi K3
  22. 54.5 Llama 4 Maverick
  23. 53.7 Claude Opus 5.5
  24. 52.7 Claude 3.5 Sonnet
  25. 52.7 Claude 3.7 Sonnet
  26. 52.7 Qwen3.7-Max
  27. 52.7 Gemini 3.5 Flash-Lite
  28. 50.9 GLM-5.3
  29. 47.3 Kimi K2.5
  30. 47.3 MiniMax-M3
  31. 47.3 GPT-6 Astra
  32. 45.5 MiMo-V2.5-Pro
  33. 43.6 stealth/ox-alpha
  34. 41.8 Nemotron 3 Ultra
  35. 41.8 DeepSeek V4.1 Flash
  36. 41.8 GPT-6 Sol
  37. 40.0 openai/gpt-6-luna-pro
  38. 40.0 Claude Opus 4.1
  39. 40.0 DeepSeek V4 Flash 0731
  40. 40.0 Muse Spark 1.2
  41. 40.0 GPT-6 Luna
  42. 38.2 openai/gpt-6-sol-pro
  43. 36.4 Llama 4 Scout
  44. 36.4 GPT-5.1 Instant
  45. 36.4 GPT-5.2 Codex
  46. 34.5 Llama 3.1-8B
  47. 34.5 GPT-5.6 Terra
  48. 32.7 openrouter/hunter-alpha
  49. 32.7 Claude Opus 4
  50. 32.7 DeepSeek V4 Pro 0813
  51. 30.9 GPT-5-Codex
  52. 30.9 Gemini 3 Pro
  53. 30.9 MiMo-V2.5
  54. 29.1 openrouter/healer-alpha
  55. 29.1 GPT-5.1-Codex
  56. 27.3 GPT-5.2
  57. 27.3 Nemotron 3 Nano 30B-A3B
  58. 27.3 GPT-5.3 Instant
  59. 27.3 GPT-5.5
  60. 27.3 GPT-5.6 Sol
  61. 25.4 Claude Sonnet 4
  62. 25.4 GPT-5.1
  63. 25.4 Step 3.5 Flash
  64. 25.4 GPT-5.4
  65. 25.4 GPT-5.4 Mini
  66. 25.4 GPT-5.6 Luna
  67. 23.6 o3
  68. 23.6 Gemini 2.5 Pro
  69. 23.6 Laguna S 2.1
  70. 23.6 Gemini 3.7 Flash
  71. 21.8 GPT-5
  72. 21.8 Gemini 3.1 Pro
  73. 21.8 Gemma 4 31B IT
  74. 20.0 Qwen3-Coder-480B-A35B
  75. 20.0 Kimi K2
  76. 20.0 GLM-5
  77. 20.0 Trinity Large Thinking
  78. 18.2 Claude 3 Haiku
  79. 18.2 GLM-5.2
  80. 18.2 Gemini 3.6 Flash
  81. 16.4 GPT-4o
  82. 16.4 MiMo-V2-Flash
  83. 16.4 MiniMax-M2.5
  84. 16.4 GPT-5.4 Nano
  85. 16.4 DeepSeek-V4-Flash
  86. 14.6 baidu/ernie-4.5-vl-424b-a47b
  87. 14.6 openai/gpt-5.5-chat
  88. 14.6 GPT-4.1
  89. 14.6 GLM-4.5
  90. 14.6 GPT-5 Chat
  91. 14.6 Gemini 2.5 Flash
  92. 14.6 Grok 4.1 Fast
  93. 14.6 DeepSeek-V3.2
  94. 14.6 GPT-5.2 Instant
  95. 14.6 GPT-5.3 Codex
  96. 14.6 Gemma 4 26B A4B
  97. 14.6 DeepSeek-V4-Pro
  98. 12.7 Gemini 2.0 Flash
  99. 12.7 DeepSeek-R1
  100. 12.7 ERNIE-4.5-300B-A47B
  101. 12.7 INTELLECT-3
  102. 12.7 Tencent Hy3 preview
  103. 10.9 o4-mini
  104. 10.9 Doubao-Seed-1.6
  105. 10.9 Gemini 3 Flash
  106. 9.09 Gemini 3.1 Flash-Lite
  107. 9.09 Gemini 3.5 Flash
  108. 7.27 GPT-4o mini
  109. 7.27 Qwen3-235B-A22B
  110. 7.27 Jamba 1.7 Large
  111. 7.27 nvidia-nemotron-nano-9b-v2
  112. 5.45 deepseek/deepseek-chat
  113. 3.64 Gemma 3 27B
  114. 3.64 gpt-oss-120b
  115. 3.64 Mistral Large 3
  1. 1 Claude Opus 4.8 none 96.4%
  2. 2 Claude Sonnet 4.6 none 94.5%
  3. 3 Claude Opus 4.6 none 92.7%
  4. 4 Claude Opus 4.5 high 90.9%
  5. 5 Claude Haiku 4.5 high 87.3%
  6. 6 Qwen 3.8 Max minimal 87.3%
  7. 7 Claude Opus 4.7 none 80%
  8. 8 Qwen 3.8 27B none 76.4%
  9. 9 Claude Fable 5.1 low 71.7%
  10. 10 Grok 4.20 Multi-agent xhigh 67.3%
  11. 11 Claude Sonnet 4.5 high 65.5%
  12. 12 Qwen3.5 397B-A17B none 65.5%
  13. 13 Nemotron 3 Super xhigh 65.5%
  14. 14 Claude Sonnet 5 low 65.5%
  15. 15 Grok 4.6 minimal 63.6%
  16. 16 Grok 4.20 low 61.8%
  17. 17 Grok 4.3 minimal 61.8%
  18. 18 Grok 4.5 high 60%
  19. 19 Claude Opus 5 low 60%
  20. 20 Claude 3.5 Haiku 58.2%
  21. 21 Kimi K3 minimal 58.2%
  22. 22 Llama 4 Maverick 54.5%
  23. 23 Claude Opus 5.5 low 53.7%
  24. 24 Claude 3.5 Sonnet 52.7%
  25. 25 Claude 3.7 Sonnet thinking 52.7%
  26. 26 Qwen3.7-Max none 52.7%
  27. 27 Gemini 3.5 Flash-Lite minimal 52.7%
  28. 28 GLM-5.3 minimal 50.9%
  29. 29 Kimi K2.5 none 47.3%
  30. 30 MiniMax-M3 xhigh 47.3%
  31. 31 GPT-6 Astra max 47.3%
  32. 32 MiMo-V2.5-Pro xhigh 45.5%
  33. 33 stealth/ox-alpha low 43.6%
  34. 34 Nemotron 3 Ultra none 41.8%
  35. 35 DeepSeek V4.1 Flash max 41.8%
  36. 36 GPT-6 Sol max 41.8%
  37. 37 openai/gpt-6-luna-pro none 40%
  38. 38 Claude Opus 4.1 none 40%
  39. 39 DeepSeek V4 Flash 0731 xhigh 40%
  40. 40 Muse Spark 1.2 minimal 40%
  41. 41 GPT-6 Luna none 40%
  42. 42 openai/gpt-6-sol-pro none 38.2%
  43. 43 Llama 4 Scout 36.4%
  44. 44 GPT-5.1 Instant 36.4%
  45. 45 GPT-5.2 Codex low 36.4%
  46. 46 Llama 3.1-8B 34.5%
  47. 47 GPT-5.6 Terra max 34.5%
  48. 48 openrouter/hunter-alpha none 32.7%
  49. 49 Claude Opus 4 32.7%
  50. 50 DeepSeek V4 Pro 0813 max 32.7%
  51. 51 GPT-5-Codex 30.9%
  52. 52 Gemini 3 Pro low 30.9%
  53. 53 MiMo-V2.5 none 30.9%
  54. 54 openrouter/healer-alpha none 29.1%
  55. 55 GPT-5.1-Codex 29.1%
  56. 56 GPT-5.2 none 27.3%
  57. 57 Nemotron 3 Nano 30B-A3B none 27.3%
  58. 58 GPT-5.3 Instant 27.3%
  59. 59 GPT-5.5 none 27.3%
  60. 60 GPT-5.6 Sol max 27.3%
  61. 61 Claude Sonnet 4 none 25.4%
  62. 62 GPT-5.1 25.4%
  63. 63 Step 3.5 Flash minimal 25.4%
  64. 64 GPT-5.4 none 25.4%
  65. 65 GPT-5.4 Mini xhigh 25.4%
  66. 66 GPT-5.6 Luna low 25.4%
  67. 67 o3 low 23.6%
  68. 68 Gemini 2.5 Pro Jun 2025 23.6%
  69. 69 Laguna S 2.1 xhigh 23.6%
  70. 70 Gemini 3.7 Flash minimal 23.6%
  71. 71 GPT-5 21.8%
  72. 72 Gemini 3.1 Pro low 21.8%
  73. 73 Gemma 4 31B IT high 21.8%
  74. 74 Qwen3-Coder-480B-A35B 20%
  75. 75 Kimi K2 Sep 2025 20%
  76. 76 GLM-5 high 20%
  77. 77 Trinity Large Thinking xhigh 20%
  78. 78 Claude 3 Haiku 18.2%
  79. 79 GLM-5.2 xhigh 18.2%
  80. 80 Gemini 3.6 Flash minimal 18.2%
  81. 81 GPT-4o Aug 2024 16.4%
  82. 82 MiMo-V2-Flash none 16.4%
  83. 83 MiniMax-M2.5 high 16.4%
  84. 84 GPT-5.4 Nano none 16.4%
  85. 85 DeepSeek-V4-Flash none 16.4%
  86. 86 baidu/ernie-4.5-vl-424b-a47b xhigh 14.6%
  87. 87 openai/gpt-5.5-chat 14.6%
  88. 88 GPT-4.1 14.6%
  89. 89 GLM-4.5 xhigh 14.6%
  90. 90 GPT-5 Chat 14.6%
  91. 91 Gemini 2.5 Flash Sep 2025 14.6%
  92. 92 Grok 4.1 Fast high 14.6%
  93. 93 DeepSeek-V3.2 high 14.6%
  94. 94 GPT-5.2 Instant 14.6%
  95. 95 GPT-5.3 Codex low 14.6%
  96. 96 Gemma 4 26B A4B xhigh 14.6%
  97. 97 DeepSeek-V4-Pro none 14.6%
  98. 98 Gemini 2.0 Flash Feb 2025 12.7%
  99. 99 DeepSeek-R1 May 2025, none 12.7%
  100. 100 ERNIE-4.5-300B-A47B 12.7%
  101. 101 INTELLECT-3 low 12.7%
  102. 102 Tencent Hy3 preview xhigh 12.7%
  103. 103 o4-mini high 10.9%
  104. 104 Doubao-Seed-1.6 none 10.9%
  105. 105 Gemini 3 Flash high 10.9%
  106. 106 Gemini 3.1 Flash-Lite 9.09%
  107. 107 Gemini 3.5 Flash minimal 9.09%
  108. 108 GPT-4o mini 7.27%
  109. 109 Qwen3-235B-A22B none 7.27%
  110. 110 Jamba 1.7 Large 7.27%
  111. 111 nvidia-nemotron-nano-9b-v2 xhigh 7.27%
  112. 112 deepseek/deepseek-chat 5.45%
  113. 113 Gemma 3 27B 3.64%
  114. 114 gpt-oss-120b low 3.64%
  115. 115 Mistral Large 3 3.64%
#EntryScoreSettingDateSource
1 Claude Opus 4.8 96.4% none May 2026 Official leaderboard
2 Claude Sonnet 4.6 94.5% none Feb 2026 Official leaderboard
3 Claude Opus 4.6 92.7% none Feb 2026 Official leaderboard
4 Claude Opus 4.5 90.9% high Nov 2025 Official leaderboard
5 Claude Haiku 4.5 87.3% high Oct 2025 Official leaderboard
6 Qwen 3.8 Max 87.3% minimal Aug 2026 Official leaderboard
7 Claude Opus 4.7 80% none Apr 2026 Official leaderboard
8 Qwen 3.8 27B 76.4% none Aug 2026 Official leaderboard
9 Claude Fable 5.1 71.7% low Sep 2026 Official leaderboard
10 Grok 4.20 Multi-agent 67.3% xhigh Mar 2026 Official leaderboard
11 Claude Sonnet 4.5 65.5% high Sep 2025 Official leaderboard
12 Qwen3.5 397B-A17B 65.5% none Feb 2026 Official leaderboard
13 Nemotron 3 Super 65.5% xhigh Mar 2026 Official leaderboard
14 Claude Sonnet 5 65.5% low Jun 2026 Official leaderboard
15 Grok 4.6 63.6% minimal Aug 2026 Official leaderboard
16 Grok 4.20 61.8% low Feb 2026 Official leaderboard
17 Grok 4.3 61.8% minimal Apr 2026 Official leaderboard
18 Grok 4.5 60% high Jul 2026 Official leaderboard
19 Claude Opus 5 60% low Jul 2026 Official leaderboard
20 Claude 3.5 Haiku 58.2% Oct 2024 Official leaderboard
21 Kimi K3 58.2% minimal Jul 2026 Official leaderboard
22 Llama 4 Maverick 54.5% Apr 2025 Official leaderboard
23 Claude Opus 5.5 53.7% low Sep 2026 Official leaderboard
24 Claude 3.5 Sonnet 52.7% Jun 2024 Official leaderboard
25 Claude 3.7 Sonnet 52.7% thinking Feb 2025 Official leaderboard
26 Qwen3.7-Max 52.7% none May 2026 Official leaderboard
27 Gemini 3.5 Flash-Lite 52.7% minimal Jul 2026 Official leaderboard
28 GLM-5.3 50.9% minimal Aug 2026 Official leaderboard
29 Kimi K2.5 47.3% none Jan 2026 Official leaderboard
30 MiniMax-M3 47.3% xhigh Jun 2026 Official leaderboard
31 GPT-6 Astra 47.3% max Sep 2026 Official leaderboard
32 MiMo-V2.5-Pro 45.5% xhigh Apr 2026 Official leaderboard
33 stealth/ox-alpha 43.6% low Official leaderboard
34 Nemotron 3 Ultra 41.8% none Jun 2026 Official leaderboard
35 DeepSeek V4.1 Flash 41.8% max Sep 2026 Official leaderboard
36 GPT-6 Sol 41.8% max Sep 2026 Official leaderboard
37 openai/gpt-6-luna-pro 40% none Official leaderboard
38 Claude Opus 4.1 40% none Aug 2025 Official leaderboard
39 DeepSeek V4 Flash 0731 40% xhigh Jul 2026 Official leaderboard
40 Muse Spark 1.2 40% minimal Aug 2026 Official leaderboard
41 GPT-6 Luna 40% none Sep 2026 Official leaderboard
42 openai/gpt-6-sol-pro 38.2% none Official leaderboard
43 Llama 4 Scout 36.4% Apr 2025 Official leaderboard
44 GPT-5.1 Instant 36.4% Nov 2025 Official leaderboard
45 GPT-5.2 Codex 36.4% low Dec 2025 Official leaderboard
46 Llama 3.1-8B 34.5% Jul 2024 Official leaderboard
47 GPT-5.6 Terra 34.5% max Jul 2026 Official leaderboard
48 openrouter/hunter-alpha 32.7% none Official leaderboard
49 Claude Opus 4 32.7% May 2025 Official leaderboard
50 DeepSeek V4 Pro 0813 32.7% max Aug 2026 Official leaderboard
51 GPT-5-Codex 30.9% Sep 2025 Official leaderboard
52 Gemini 3 Pro 30.9% low Nov 2025 Official leaderboard
53 MiMo-V2.5 30.9% none Apr 2026 Official leaderboard
54 openrouter/healer-alpha 29.1% none Official leaderboard
55 GPT-5.1-Codex 29.1% Nov 2025 Official leaderboard
56 GPT-5.2 27.3% none Dec 2025 Official leaderboard
57 Nemotron 3 Nano 30B-A3B 27.3% none Dec 2025 Official leaderboard
58 GPT-5.3 Instant 27.3% Mar 2026 Official leaderboard
59 GPT-5.5 27.3% none Apr 2026 Official leaderboard
60 GPT-5.6 Sol 27.3% max Jul 2026 Official leaderboard
61 Claude Sonnet 4 25.4% none May 2025 Official leaderboard
62 GPT-5.1 25.4% Nov 2025 Official leaderboard
63 Step 3.5 Flash 25.4% minimal Feb 2026 Official leaderboard
64 GPT-5.4 25.4% none Mar 2026 Official leaderboard
65 GPT-5.4 Mini 25.4% xhigh Mar 2026 Official leaderboard
66 GPT-5.6 Luna 25.4% low Jul 2026 Official leaderboard
67 o3 23.6% low Apr 2025 Official leaderboard
68 Gemini 2.5 Pro Jun 2025 23.6% Jun 2025 Official leaderboard
69 Laguna S 2.1 23.6% xhigh Jul 2026 Official leaderboard
70 Gemini 3.7 Flash 23.6% minimal Aug 2026 Official leaderboard
71 GPT-5 21.8% Aug 2025 Official leaderboard
72 Gemini 3.1 Pro 21.8% low Feb 2026 Official leaderboard
73 Gemma 4 31B IT 21.8% high Apr 2026 Official leaderboard
74 Qwen3-Coder-480B-A35B 20% Jul 2025 Official leaderboard
75 Kimi K2 Sep 2025 20% Sep 2025 Official leaderboard
76 GLM-5 20% high Feb 2026 Official leaderboard
77 Trinity Large Thinking 20% xhigh Apr 2026 Official leaderboard
78 Claude 3 Haiku 18.2% Mar 2024 Official leaderboard
79 GLM-5.2 18.2% xhigh Jun 2026 Official leaderboard
80 Gemini 3.6 Flash 18.2% minimal Jul 2026 Official leaderboard
81 GPT-4o Aug 2024 16.4% Aug 2024 Official leaderboard
82 MiMo-V2-Flash 16.4% none Dec 2025 Official leaderboard
83 MiniMax-M2.5 16.4% high Feb 2026 Official leaderboard
84 GPT-5.4 Nano 16.4% none Mar 2026 Official leaderboard
85 DeepSeek-V4-Flash 16.4% none Apr 2026 Official leaderboard
86 baidu/ernie-4.5-vl-424b-a47b 14.6% xhigh Official leaderboard
87 openai/gpt-5.5-chat 14.6% Official leaderboard
88 GPT-4.1 14.6% Apr 2025 Official leaderboard
89 GLM-4.5 14.6% xhigh Aug 2025 Official leaderboard
90 GPT-5 Chat 14.6% Aug 2025 Official leaderboard
91 Gemini 2.5 Flash Sep 2025 14.6% Sep 2025 Official leaderboard
92 Grok 4.1 Fast 14.6% high Nov 2025 Official leaderboard
93 DeepSeek-V3.2 14.6% high Dec 2025 Official leaderboard
94 GPT-5.2 Instant 14.6% Dec 2025 Official leaderboard
95 GPT-5.3 Codex 14.6% low Feb 2026 Official leaderboard
96 Gemma 4 26B A4B 14.6% xhigh Apr 2026 Official leaderboard
97 DeepSeek-V4-Pro 14.6% none Apr 2026 Official leaderboard
98 Gemini 2.0 Flash Feb 2025 12.7% Feb 2025 Official leaderboard
99 DeepSeek-R1 May 2025 12.7% none May 2025 Official leaderboard
100 ERNIE-4.5-300B-A47B 12.7% Jun 2025 Official leaderboard
101 INTELLECT-3 12.7% low Nov 2025 Official leaderboard
102 Tencent Hy3 preview 12.7% xhigh Apr 2026 Official leaderboard
103 o4-mini 10.9% high Apr 2025 Official leaderboard
104 Doubao-Seed-1.6 10.9% none Jun 2025 Official leaderboard
105 Gemini 3 Flash 10.9% high Dec 2025 Official leaderboard
106 Gemini 3.1 Flash-Lite 9.09% Mar 2026 Official leaderboard
107 Gemini 3.5 Flash 9.09% minimal May 2026 Official leaderboard
108 GPT-4o mini 7.27% Jul 2024 Official leaderboard
109 Qwen3-235B-A22B 7.27% none Apr 2025 Official leaderboard
110 Jamba 1.7 Large 7.27% Jul 2025 Official leaderboard
111 nvidia-nemotron-nano-9b-v2 7.27% xhigh Aug 2025 Official leaderboard
112 deepseek/deepseek-chat 5.45% Official leaderboard
113 Gemma 3 27B 3.64% Mar 2025 Official leaderboard
114 gpt-oss-120b 3.64% low Aug 2025 Official leaderboard
115 Mistral Large 3 3.64% Dec 2025 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.8
none
96.4%
Claude Sonnet 4.6
none
94.5%
Claude Opus 4.6
none
92.7%
Claude Opus 4.5
none high
90.9%
Claude Haiku 4.5
none
87.3%
Qwen 3.8 Max
minimal max
87.3%
Claude Opus 4.7
none max
80%
Qwen 3.8 27B
none max
76.4%
Claude Fable 5.1
low max
71.7%
Grok 4.20 Multi-agent
low
67.3%
Claude Sonnet 4.5
none high
65.5%
Qwen3.5 397B-A17B
none high
65.5%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 none 96.4%
Claude Sonnet 4.6 none 94.5%
Claude Opus 4.6 none 92.7%
Claude Sonnet 4.6 high 92.7%
Claude Opus 4.8 xhigh 92.7%
Claude Opus 4.5 high 90.9%
Claude Opus 4.6 high 89.1%
Claude Haiku 4.5 high 87.3%
Qwen 3.8 Max minimal 87.3%
Claude Haiku 4.5 none 81.8%
Claude Opus 4.5 none 80%
Claude Opus 4.7 none 80%
Qwen 3.8 27B none 76.4%
Claude Fable 5.1 low 71.7%
Grok 4.20 Multi-agent xhigh 67.3%
Grok 4.20 Multi-agent low 67.3%
Claude Sonnet 4.5 high 65.5%
Qwen3.5 397B-A17B none 65.5%
Nemotron 3 Super xhigh 65.5%
Claude Sonnet 5 low 65.5%
Grok 4.6 minimal 63.6%
Grok 4.20 low 61.8%
Grok 4.3 minimal 61.8%
Claude Sonnet 5 max 61.8%
Claude Fable 5.1 max 61.1%
Claude Opus 4.7 max 60%
Grok 4.5 high 60%
Claude Opus 5 low 60%
Grok 4.6 max 60%
Claude 3.5 Haiku 58.2%
Grok 4.20 xhigh 58.2%
Kimi K3 minimal 58.2%
Claude Opus 5 xhigh 58.2%
Grok 4.3 xhigh 56.4%
Llama 4 Maverick 54.5%
Grok 4.5 low 54.5%
Claude Opus 5.5 low 53.7%
Claude 3.5 Sonnet 52.7%
Claude 3.7 Sonnet thinking 52.7%
Claude Sonnet 4.5 none 52.7%
Qwen3.5 397B-A17B high 52.7%
Qwen3.7-Max none 52.7%
Gemini 3.5 Flash-Lite minimal 52.7%
GLM-5.3 minimal 50.9%
GLM-5.3 max 50.9%
Gemini 3.5 Flash-Lite xhigh 49.1%
Kimi K2.5 none 47.3%
MiniMax-M3 xhigh 47.3%
GPT-6 Astra max 47.3%
Claude 3.7 Sonnet 45.5%
MiMo-V2.5-Pro xhigh 45.5%
MiniMax-M3 none 45.5%
Claude Opus 5.5 max 44.4%
Kimi K3 xhigh 43.6%
Nemotron 3 Ultra none 41.8%
DeepSeek V4.1 Flash max 41.8%
DeepSeek V4.1 Flash low 41.8%
GPT-6 Sol max 41.8%
Claude Opus 4.1 none 40%
DeepSeek V4 Flash 0731 xhigh 40%
Muse Spark 1.2 minimal 40%
GPT-6 Luna none 40%
Nemotron 3 Super none 38.2%
Qwen3.7-Max xhigh 38.2%
Llama 4 Scout 36.4%
GPT-5.1 Instant 36.4%
GPT-5.2 Codex low 36.4%
Llama 3.1-8B 34.5%
Claude Opus 4.1 high 34.5%
GPT-5.6 Terra max 34.5%
GPT-6 Luna max 34.5%
Claude Opus 4 32.7%
Muse Spark 1.2 xhigh 32.7%
DeepSeek V4 Pro 0813 max 32.7%
GPT-6 Sol none 32.7%
GPT-5-Codex 30.9%
Gemini 3 Pro low 30.9%
Kimi K2.5 high 30.9%
MiMo-V2.5 none 30.9%
Nemotron 3 Ultra xhigh 30.9%
GPT-5.1-Codex 29.1%
GPT-5.2 Codex high 29.1%
MiMo-V2.5-Pro none 29.1%
GPT-5.6 Terra low 29.1%
GPT-6 Astra low 29.1%
GPT-5.2 none 27.3%
Nemotron 3 Nano 30B-A3B none 27.3%
GPT-5.3 Instant 27.3%
GPT-5.5 none 27.3%
GPT-5.6 Sol max 27.3%
Claude Sonnet 4 none 25.4%
GPT-5.1 25.4%
GPT-5.2 Codex xhigh 25.4%
Step 3.5 Flash minimal 25.4%
GPT-5.4 none 25.4%
GPT-5.4 Mini xhigh 25.4%
GPT-5.4 Mini high 25.4%
GPT-5.6 Luna low 25.4%
GPT-5.6 Sol low 25.4%
DeepSeek V4 Pro 0813 low 25.4%
o3 low 23.6%
Claude Sonnet 4 high 23.6%
Gemini 2.5 Pro Jun 2025 23.6%
Gemini 3 Pro high 23.6%
GPT-5.2 high 23.6%
GPT-5.4 Mini none 23.6%
Laguna S 2.1 xhigh 23.6%
Gemini 3.7 Flash minimal 23.6%
Gemini 3.7 Flash max 23.6%
GPT-5 21.8%
Gemini 3.1 Pro low 21.8%
GPT-5.4 xhigh 21.8%
Gemma 4 31B IT high 21.8%
DeepSeek V4 Flash 0731 none 21.8%
Qwen 3.8 Max max 21.8%
Qwen3-Coder-480B-A35B 20%
Kimi K2 Sep 2025 20%
Step 3.5 Flash xhigh 20%
GLM-5 high 20%
Trinity Large Thinking xhigh 20%
MiMo-V2.5 xhigh 20%
Laguna S 2.1 none 20%
Qwen 3.8 27B max 20%
Claude 3 Haiku 18.2%
Gemini 3.1 Pro high 18.2%
Trinity Large Thinking minimal 18.2%
GLM-5.2 xhigh 18.2%
GPT-5.6 Luna max 18.2%
Gemini 3.6 Flash minimal 18.2%
GPT-4o Aug 2024 16.4%
MiMo-V2-Flash none 16.4%
GLM-5 none 16.4%
MiniMax-M2.5 high 16.4%
GPT-5.4 Nano none 16.4%
DeepSeek-V4-Flash none 16.4%
Gemini 3.6 Flash xhigh 16.4%
GPT-4o Aug 2024 14.6%
GPT-4.1 14.6%
GLM-4.5 xhigh 14.6%
GPT-5 Chat 14.6%
Gemini 2.5 Flash Sep 2025 14.6%
Grok 4.1 Fast high 14.6%
DeepSeek-V3.2 high 14.6%
GPT-5.2 Instant 14.6%
GPT-5.3 Codex low 14.6%
GPT-5.4 Nano xhigh 14.6%
Gemma 4 26B A4B xhigh 14.6%
DeepSeek-V4-Pro none 14.6%
Gemini 2.0 Flash Feb 2025 12.7%
DeepSeek-R1 May 2025 none 12.7%
ERNIE-4.5-300B-A47B 12.7%
INTELLECT-3 low 12.7%
DeepSeek-V3.2 none 12.7%
GPT-5.3 Codex high 12.7%
MiniMax-M2.5 low 12.7%
Tencent Hy3 preview xhigh 12.7%
Tencent Hy3 preview none 12.7%
DeepSeek-V4-Flash xhigh 12.7%
o4-mini high 10.9%
Doubao-Seed-1.6 none 10.9%
Grok 4.1 Fast none 10.9%
INTELLECT-3 high 10.9%
Nemotron 3 Nano 30B-A3B xhigh 10.9%
Gemini 3 Flash high 10.9%
GPT-5.3 Codex xhigh 10.9%
GPT-5.4 Nano high 10.9%
Gemma 4 31B IT none 10.9%
DeepSeek-V4-Pro xhigh 10.9%
GLM-5.2 none 10.9%
o4-mini low 9.09%
MiMo-V2-Flash high 9.09%
Gemini 3.1 Flash-Lite 9.09%
Gemma 4 26B A4B none 9.09%
Gemini 3.5 Flash minimal 9.09%
Gemini 3.5 Flash xhigh 9.09%
GPT-4o mini 7.27%
Qwen3-235B-A22B none 7.27%
DeepSeek-R1 May 2025 xhigh 7.27%
Doubao-Seed-1.6 high 7.27%
Jamba 1.7 Large 7.27%
nvidia-nemotron-nano-9b-v2 xhigh 7.27%
nvidia-nemotron-nano-9b-v2 none 7.27%
Gemini 3 Flash none 7.27%
GPT-4o mini 5.45%
Qwen3-235B-A22B xhigh 5.45%
Gemma 3 27B 3.64%
GLM-4.5 none 3.64%
gpt-oss-120b low 3.64%
gpt-oss-120b high 3.64%
Mistral Large 3 3.64%

4 models have held number one since its release in Feb 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Feb 2026, by Claude Opus 4.5
Claude Opus 4.8 96.4%

The best score passed 85% in Oct 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Opus 4.5 Claude Sonnet 4.5 Claude 3.5 Haiku Claude 3.5 Sonnet Claude 3 Haiku 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Claude Haiku 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%
BullshitBench Passed 85% in Feb 2026, by Claude Opus 4.5
Claude Opus 4.8 96.4%
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%
Replaced by BullshitBench V2

More in Factuality

All 70 in Factuality
  • SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. Qwen3.7-Plus 81.7%
  • HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy. Keye-VL-2.0-30B-A3B 73.5%
  • SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. Gemini 3.1 Pro 77.5%
  • GaRAGe In use Models write long-form RAG answers to 2,366 questions with human-annotated grounding passages. Scoring checks whether answers use the relevant passages. Nemotron 3.5 Bolt E4B 40.7%
  • Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. Kimi K3.1 Pro 8.30%
  • RAGTruth In use A word-level annotated corpus of hallucinations in retrieval-augmented generation answers. It is used to train and evaluate hallucination detection and RAG faithfulness. Nemotron 3.5 Bolt E4B 77.9%