Benchmark Almanac

Hermes Bench

Claude Opus 5.5 76.7% leads since Oct 2026 Not in model cards

Agents in Hermes Agent do 150 tasks across 25 categories, such as research, diagrams, memory, tool use and safety. Graders check the files and state left behind.

Launched 2026-10-06 as part of the Hermes Index. 24 tasks are multi-turn.

Made by Nous Research Released Oct 2026 General assistants Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
Leaderboard portal.nousresearch.com
18
Entries on its own leaderboard, 17 models we track

Claude Opus 5.5 leads with 76.7%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

18 entries. Scroll sideways for the rest.

  1. 76.7 Claude Opus 5.5
  2. 74.4 Grok 4.7
  3. 73.9 GPT-6.1 Sol
  4. 73.7 GPT-6 Astra
  5. 73.5 GPT-6 Sol
  6. 73.5 Claude Sonnet 5.5
  7. 71.8 GLM-5.3
  8. 71.5 Kimi K3
  9. 71.0 Claude Haiku 5.5
  10. 70.7 DeepSeek V4.1 Flash
  11. 70.5 Qwen 3.8 Max
  12. 70.5 GPT-6 Luna
  13. 69.2 Step 5 Preview
  14. 68.9 GLM-5.3-Flash
  15. 63.4 Nemotron Super 3.5
  16. 62.2 Gemini 3.8 Flash
  17. 58.6 Tencent Hy4 preview
  18. 53.8 Ling-3.0-flash
  1. 1 Claude Opus 5.5 76.7%
  2. 2 Grok 4.7 74.4%
  3. 3 GPT-6.1 Sol 73.9%
  4. 4 GPT-6 Astra 73.7%
  5. 5 GPT-6 Sol 73.5%
  6. 6 Claude Sonnet 5.5 73.5%
  7. 7 GLM-5.3 71.8%
  8. 8 Kimi K3 71.5%
  9. 9 Claude Haiku 5.5 71%
  10. 10 DeepSeek V4.1 Flash 70.7%
  11. 11 Qwen 3.8 Max 70.5%
  12. 12 GPT-6 Luna 70.5%
  13. 13 Step 5 Preview 69.2%
  14. 14 GLM-5.3-Flash 68.9%
  15. 15 Nemotron Super 3.5 63.4%
  16. 16 Gemini 3.8 Flash 62.2%
  17. 17 Tencent Hy4 preview 58.6%
  18. 18 Ling-3.0-flash 53.8%
#EntryScoreSettingDateSource
1 Claude Opus 5.5 76.7% Hermes Agent Sep 2026 Official leaderboard
2 Grok 4.7 74.4% Hermes Agent Sep 2026 Official leaderboard
3 GPT-6.1 Sol 73.9% Hermes Agent Sep 2026 Official leaderboard
4 GPT-6 Astra 73.7% Hermes Agent Sep 2026 Official leaderboard
5 GPT-6 Sol 73.5% Hermes Agent Sep 2026 Official leaderboard
6 Claude Sonnet 5.5 73.5% Hermes Agent Sep 2026 Official leaderboard
7 GLM-5.3 71.8% Hermes Agent Aug 2026 Official leaderboard
8 Kimi K3 71.5% Hermes Agent Jul 2026 Official leaderboard
9 Claude Haiku 5.5 71% Hermes Agent Oct 2026 Official leaderboard
10 DeepSeek V4.1 Flash 70.7% Hermes Agent Sep 2026 Official leaderboard
11 Qwen 3.8 Max 70.5% Hermes Agent Aug 2026 Official leaderboard
12 GPT-6 Luna 70.5% Hermes Agent Sep 2026 Official leaderboard
13 Step 5 Preview 69.2% Hermes Agent Sep 2026 Official leaderboard
14 GLM-5.3-Flash 68.9% Hermes Agent Aug 2026 Official leaderboard
15 Nemotron Super 3.5 63.4% Hermes Agent Official leaderboard
16 Gemini 3.8 Flash 62.2% Hermes Agent Sep 2026 Official leaderboard
17 Tencent Hy4 preview 58.6% Hermes Agent Aug 2026 Official leaderboard
18 Ling-3.0-flash 53.8% Hermes Agent Aug 2026 Official leaderboard

The best score, 76.7%, costs $0.52 a task

Runs on the cost frontier score higher than any run that's cheaper. GLM-5.3 comes within 4.9 points of the best for $0.18 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Moonshot AI Ant Group Z.ai Anthropic OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 5.5 GLM-5.3 Claude Haiku 5.5 Ling-3.0-flash 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 5.5 Claude Haiku 5.5
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 Hermes Agent 76.7% $0.52
Grok 4.7 Hermes Agent 74.4% $0.69
GPT-6.1 Sol Hermes Agent 73.9% $0.26
GPT-6 Astra Hermes Agent 73.7% $1.52
GPT-6 Sol Hermes Agent 73.5% $0.26
Claude Sonnet 5.5 Hermes Agent 73.5% $0.25
GLM-5.3 Hermes Agent 71.8% $0.18
Kimi K3 Hermes Agent 71.5% $0.18
Claude Haiku 5.5 Hermes Agent 71% $0.01
DeepSeek V4.1 Flash Hermes Agent 70.7% $0.02
Qwen 3.8 Max Hermes Agent 70.5% $0.32
GPT-6 Luna Hermes Agent 70.5% $0.01
Step 5 Preview Hermes Agent 69.2% $0.17
GLM-5.3-Flash Hermes Agent 68.9% $0.02
Gemini 3.8 Flash Hermes Agent 62.2% $0.64
Tencent Hy4 preview Hermes Agent 58.6% $0.09
Ling-3.0-flash Hermes Agent 53.8% $0.004

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader
Claude Opus 5.5 76.7%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5 GLM-5.3 0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Opus 5.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Tool use

All 125 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Muse Spark 1.1 88.1%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Claw-Eval In use 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. Claude Sonnet 4.6 81.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%