Benchmark Almanac

PinchBench 2.0

Claude Opus 4.8 93.5% leads since May 2026 Not in model cards

Models drive an OpenClaw agent through 148 real-world tasks, including data and log analysis, meeting notes, DevOps, PDFs, research and writing. Graded by checks and a Haiku judge.

Up from about 25 tasks in v1.x. The default judge changed to Claude Haiku, so scores do not compare with v1. Released 2026-05-06.

Made by Kilo AI Released May 2026 General assistants All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
pinchbench/skill 1,373 stars Paper Leaderboard
60
Entries on its own leaderboard, 57 models we track

Claude Opus 4.8 leads with 93.5%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

60 entries. Scroll sideways for the rest.

  1. 93.5 Claude Opus 4.8
  2. 92.5 Qwen3.7-Max
  3. 90.3 Nemotron 3.5 Bolt 74B-A9B
  4. 89.9 Nemotron 3 Ultra
  5. 89.7 MiMo-V2.5
  6. 88.9 Grok Build 0.1
  7. 88.7 GPT-5.6 Luna
  8. 88.1 Qwen 3.6 Flash
  9. 87.8 nemotron-3.5-bolt-74b-a9b-24gb
  10. 87.5 MiMo-V2.5-Pro
  11. 87.0 GLM-5.2
  12. 86.4 Nemotron 3.5 Lightning 30B-A3B
  13. 84.2 GPT-5.6 Sol
  14. 82.6 Ling-2.6-1T
  15. 81.7 DeepSeek-V4-Flash
  16. 81.0 Gemini 3.1 Pro
  17. 80.5 Gemini 3.1 Flash-Lite
  18. 80.3 Grok 4.20
  19. 79.3 Step 3.5 Flash
  20. 79.2 GPT-5.4 Mini
  21. 76.1 Kimi K2.7 Code
  22. 76.0 Claude Opus 4.7
  23. 75.9 GPT-5.6 Terra
  24. 75.7 GPT-5.4
  25. 75.5 GPT-5.5
  26. 75.2 Grok 4.5
  27. 75.0 Seed2.0 Lite (0215)
  28. 74.2 Gemini 3.5 Flash
  29. 73.7 Grok 4.3
  30. 72.5 Qwen 3.6 Plus
  31. 72.1 Gemini 3 Flash
  32. 71.8 GLM-5-Turbo
  33. 69.9 Claude Opus 4.6
  34. 69.5 fugu-ultra
  35. 69.4 Devstral 2
  36. 69.0 GPT-5.4 Nano
  37. 67.7 Claude Haiku 4.5
  38. 67.6 GLM-5V-Turbo
  39. 66.8 MiniMax-M2.7
  40. 65.7 Trinity Large Thinking
  41. 64.6 Mistral Small 4
  42. 62.7 Claude Sonnet 4.6
  43. 61.1 aion-3.0
  44. 61.1 DeepSeek-V4-Pro
  45. 60.0 GLM-5.1
  46. 56.4 Gemma 4 26B A4B
  47. 54.8 Claude Fable 5
  48. 54.6 Kimi K2.5
  49. 54.5 Mistral Large 3
  50. 53.0 Trinity Large Preview
  51. 52.7 Gemma 4 31B IT
  52. 48.8 Claude Sonnet 4
  53. 44.8 gpt-oss-120b
  54. 42.2 Nemotron 3 Super
  55. 39.5 Mercury 2
  56. 37.6 Nova 2 Lite
  57. 36.3 gpt-oss-20b
  58. 21.4 GPT-5.5 Pro
  59. 10.7 Llama 3.1-70B
  60. 3.19 Llama 4 Scout
  1. 1 Claude Opus 4.8 fast mode 93.5%
  2. 2 Qwen3.7-Max 92.5%
  3. 3 Nemotron 3.5 Bolt 74B-A9B 90.3%
  4. 4 Nemotron 3 Ultra 89.9%
  5. 5 MiMo-V2.5 89.7%
  6. 6 Grok Build 0.1 88.9%
  7. 7 GPT-5.6 Luna 88.7%
  8. 8 Qwen 3.6 Flash 88.1%
  9. 9 nemotron-3.5-bolt-74b-a9b-24gb 87.8%
  10. 10 MiMo-V2.5-Pro 87.5%
  11. 11 GLM-5.2 87%
  12. 12 Nemotron 3.5 Lightning 30B-A3B 86.4%
  13. 13 GPT-5.6 Sol 84.2%
  14. 14 Ling-2.6-1T 82.6%
  15. 15 DeepSeek-V4-Flash 81.7%
  16. 16 Gemini 3.1 Pro 81%
  17. 17 Gemini 3.1 Flash-Lite 80.5%
  18. 18 Grok 4.20 80.3%
  19. 19 Step 3.5 Flash 79.3%
  20. 20 GPT-5.4 Mini 79.2%
  21. 21 Kimi K2.7 Code 76.1%
  22. 22 Claude Opus 4.7 76%
  23. 23 GPT-5.6 Terra 75.9%
  24. 24 GPT-5.4 75.7%
  25. 25 GPT-5.5 75.5%
  26. 26 Grok 4.5 75.2%
  27. 27 Seed2.0 Lite (0215) 75%
  28. 28 Gemini 3.5 Flash 74.2%
  29. 29 Grok 4.3 73.7%
  30. 30 Qwen 3.6 Plus 72.5%
  31. 31 Gemini 3 Flash 72.1%
  32. 32 GLM-5-Turbo 71.8%
  33. 33 Claude Opus 4.6 69.9%
  34. 34 fugu-ultra 69.5%
  35. 35 Devstral 2 69.4%
  36. 36 GPT-5.4 Nano 69%
  37. 37 Claude Haiku 4.5 67.7%
  38. 38 GLM-5V-Turbo 67.6%
  39. 39 MiniMax-M2.7 66.8%
  40. 40 Trinity Large Thinking 65.7%
  41. 41 Mistral Small 4 64.6%
  42. 42 Claude Sonnet 4.6 62.7%
  43. 43 aion-3.0 61.1%
  44. 44 DeepSeek-V4-Pro 61.1%
  45. 45 GLM-5.1 60%
  46. 46 Gemma 4 26B A4B 56.4%
  47. 47 Claude Fable 5 54.8%
  48. 48 Kimi K2.5 54.6%
  49. 49 Mistral Large 3 54.5%
  50. 50 Trinity Large Preview 53%
  51. 51 Gemma 4 31B IT 52.7%
  52. 52 Claude Sonnet 4 48.8%
  53. 53 gpt-oss-120b 44.8%
  54. 54 Nemotron 3 Super 42.2%
  55. 55 Mercury 2 39.5%
  56. 56 Nova 2 Lite 37.6%
  57. 57 gpt-oss-20b 36.3%
  58. 58 GPT-5.5 Pro 21.4%
  59. 59 Llama 3.1-70B 10.7%
  60. 60 Llama 4 Scout 3.19%
#EntryScoreSettingDateSource
1 Claude Opus 4.8 93.5% fast mode, OpenClaw May 2026 Official leaderboard
2 Qwen3.7-Max 92.5% OpenClaw May 2026 Official leaderboard
3 Nemotron 3.5 Bolt 74B-A9B 90.3% OpenClaw Oct 2026 Official leaderboard
4 Nemotron 3 Ultra 89.9% OpenClaw Jun 2026 Official leaderboard
5 MiMo-V2.5 89.7% OpenClaw Apr 2026 Official leaderboard
6 Grok Build 0.1 88.9% OpenClaw May 2026 Official leaderboard
7 GPT-5.6 Luna 88.7% OpenClaw Jul 2026 Official leaderboard
8 Qwen 3.6 Flash 88.1% OpenClaw Apr 2026 Official leaderboard
9 nemotron-3.5-bolt-74b-a9b-24gb 87.8% OpenClaw Oct 2026 Official leaderboard
10 MiMo-V2.5-Pro 87.5% OpenClaw Apr 2026 Official leaderboard
11 GLM-5.2 87% OpenClaw Jun 2026 Official leaderboard
12 Nemotron 3.5 Lightning 30B-A3B 86.4% OpenClaw Aug 2026 Official leaderboard
13 GPT-5.6 Sol 84.2% OpenClaw Jul 2026 Official leaderboard
14 Ling-2.6-1T 82.6% OpenClaw Apr 2026 Official leaderboard
15 DeepSeek-V4-Flash 81.7% OpenClaw Apr 2026 Official leaderboard
16 Gemini 3.1 Pro 81% OpenClaw Feb 2026 Official leaderboard
17 Gemini 3.1 Flash-Lite 80.5% OpenClaw Mar 2026 Official leaderboard
18 Grok 4.20 80.3% OpenClaw Feb 2026 Official leaderboard
19 Step 3.5 Flash 79.3% OpenClaw Feb 2026 Official leaderboard
20 GPT-5.4 Mini 79.2% OpenClaw Mar 2026 Official leaderboard
21 Kimi K2.7 Code 76.1% OpenClaw Jun 2026 Official leaderboard
22 Claude Opus 4.7 76% OpenClaw Apr 2026 Official leaderboard
23 GPT-5.6 Terra 75.9% OpenClaw Jul 2026 Official leaderboard
24 GPT-5.4 75.7% OpenClaw Mar 2026 Official leaderboard
25 GPT-5.5 75.5% OpenClaw Apr 2026 Official leaderboard
26 Grok 4.5 75.2% OpenClaw Jul 2026 Official leaderboard
27 Seed2.0 Lite (0215) 75% OpenClaw Feb 2026 Official leaderboard
28 Gemini 3.5 Flash 74.2% OpenClaw May 2026 Official leaderboard
29 Grok 4.3 73.7% OpenClaw Apr 2026 Official leaderboard
30 Qwen 3.6 Plus 72.5% OpenClaw Mar 2026 Official leaderboard
31 Gemini 3 Flash 72.1% OpenClaw Dec 2025 Official leaderboard
32 GLM-5-Turbo 71.8% OpenClaw Mar 2026 Official leaderboard
33 Claude Opus 4.6 69.9% OpenClaw Feb 2026 Official leaderboard
34 fugu-ultra 69.5% OpenClaw Jun 2026 Official leaderboard
35 Devstral 2 69.4% OpenClaw Dec 2025 Official leaderboard
36 GPT-5.4 Nano 69% OpenClaw Mar 2026 Official leaderboard
37 Claude Haiku 4.5 67.7% OpenClaw Oct 2025 Official leaderboard
38 GLM-5V-Turbo 67.6% OpenClaw Apr 2026 Official leaderboard
39 MiniMax-M2.7 66.8% OpenClaw Mar 2026 Official leaderboard
40 Trinity Large Thinking 65.7% OpenClaw Apr 2026 Official leaderboard
41 Mistral Small 4 64.6% OpenClaw Mar 2026 Official leaderboard
42 Claude Sonnet 4.6 62.7% OpenClaw Feb 2026 Official leaderboard
43 aion-3.0 61.1% OpenClaw Jul 2026 Official leaderboard
44 DeepSeek-V4-Pro 61.1% OpenClaw Apr 2026 Official leaderboard
45 GLM-5.1 60% OpenClaw Apr 2026 Official leaderboard
46 Gemma 4 26B A4B 56.4% OpenClaw Apr 2026 Official leaderboard
47 Claude Fable 5 54.8% OpenClaw Jun 2026 Official leaderboard
48 Kimi K2.5 54.6% OpenClaw Jan 2026 Official leaderboard
49 Mistral Large 3 54.5% OpenClaw Dec 2025 Official leaderboard
50 Trinity Large Preview 53% OpenClaw Jan 2026 Official leaderboard
51 Gemma 4 31B IT 52.7% OpenClaw Apr 2026 Official leaderboard
52 Claude Sonnet 4 48.8% OpenClaw May 2025 Official leaderboard
53 gpt-oss-120b 44.8% OpenClaw Aug 2025 Official leaderboard
54 Nemotron 3 Super 42.2% OpenClaw Mar 2026 Official leaderboard
55 Mercury 2 39.5% OpenClaw Feb 2026 Official leaderboard
56 Nova 2 Lite 37.6% OpenClaw Dec 2025 Official leaderboard
57 gpt-oss-20b 36.3% OpenClaw Aug 2025 Official leaderboard
58 GPT-5.5 Pro 21.4% OpenClaw Apr 2026 Official leaderboard
59 Llama 3.1-70B 10.7% OpenClaw Jul 2024 Official leaderboard
60 Llama 4 Scout 3.19% OpenClaw Apr 2025 Official leaderboard

3 models have held number one since its release in May 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Xiaomi Corp Qwen Anthropic
Own leaderboard Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%

The best score passed 85% in Apr 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Gemini 3.1 Pro Gemini 3 Flash Claude Haiku 4.5 Claude Sonnet 4 Llama 3.1-70B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.8 Gemini 3.1 Pro

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Google DeepMind Arcee AI Kuaishou Xiaomi Corp Qwen Anthropic
PinchBench 2.0 Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%
PinchBench Passed 85% in Feb 2026, by Gemini 3 Flash
KAT-Coder-Pro V2.5 94.9%
PinchBench 2.0 Passed 85% in May 2026, by MiMo-V2.5
Claude Opus 4.8 93.5%
Replaces PinchBench

More in Tool use

All 125 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Muse Spark 1.1 88.1%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Claw-Eval In use 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. Claude Sonnet 4.6 81.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%