Benchmark Almanac

Judgemark v2.1

Claude Sonnet 4.6 94.1 leads since Feb 2026 Not in model cards

The model under test acts as a judge and scores creative writing on several literary criteria. The benchmark measures how well its scores separate stronger writers from weaker ones.

v2.1 swapped in stronger writer models to fight saturation and added judge ensembling. Now listed as a legacy board.

Made by EQ-Bench (Sam Paech) Released Jan 2025 Taste and style All tasks public Last model added Mar 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
EQ-bench/Judgemark-v2 30 stars Leaderboard eqbench.com
47
Entries on its own leaderboard, 42 models we track

Claude Sonnet 4.6 leads with 94.1 among models released by Mar 2026

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Mar 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

47 entries. Scroll sideways for the rest.

  1. 94.1 Claude Sonnet 4.6
  2. 92.9 Claude Opus 4.6
  3. 86.6 Gemini 3 Pro
  4. 85.6 GLM-5
  5. 85.5 Qwen3.5 397B-A17B
  6. 85.4 Gemini 3 Flash
  7. 85.0 GPT-5.4
  8. 84.6 Claude Sonnet 4.5
  9. 84.3 Qwen3.5-27B
  10. 82.0 GPT-5.2
  11. 82.0 Claude Sonnet 4
  12. 81.8 qwen3.5-flash-02-23
  13. 81.4 Qwen3.5-35B-A3B
  14. 81.2 gpt-5-2025-08-07:minimal-reasoning
  15. 81.1 GPT-5.4 Mini
  16. 81.0 Qwen3.5-122B-A10B
  17. 80.9 Gemini 3.1 Flash-Lite
  18. 79.7 Claude Haiku 4.5
  19. 77.5 GLM-4.7
  20. 76.3 GPT-4.1
  21. 75.8 Grok 4.1 Fast
  22. 75.1 o3
  23. 75.0 Qwen3.5-9B
  24. 74.5 gpt-5-mini-2025-08-07:minimal-reasoning
  25. 73.1 Kimi K2
  26. 72.1 Gemini 2.5 Pro
  27. 68.2 Mistral Large 3
  28. 66.3 DeepSeek-V3.2
  29. 66.2 glm-4.7-flash
  30. 64.5 GPT-5.4 Nano
  31. 64.5 MiniMax-M2.5
  32. 63.4 GLM-4.5
  33. 63.0 Gemini 2.5 Flash
  34. 61.1 gpt-5-nano-2025-08-07__5x-ensemble
  35. 59.7 Qwen3-235B-A22B-Instruct
  36. 58.5 Mistral Medium 3
  37. 57.3 GPT-4.1 mini
  38. 56.2 Qwen3-235B-A22B
  39. 56.1 o4-mini
  40. 56.0 Gemini 2.5 Flash-Lite
  41. 54.8 gpt-oss-120b
  42. 52.2 gpt-5-nano-2025-08-07:minimal-reasoning
  43. 51.6 Mistral Small 4
  44. 50.2 Mistral Small 3.2
  45. 42.8 gpt-oss-20b
  46. 34.6 Qwen3-30B-A3B-Instruct
  47. 19.2 Llama 3.1-8B
  1. 1 Claude Sonnet 4.6 94.1
  2. 2 Claude Opus 4.6 92.9
  3. 3 Gemini 3 Pro 86.6
  4. 4 GLM-5 85.6
  5. 5 Qwen3.5 397B-A17B 85.5
  6. 6 Gemini 3 Flash 85.4
  7. 7 GPT-5.4 85.0
  8. 8 Claude Sonnet 4.5 84.6
  9. 9 Qwen3.5-27B 84.3
  10. 10 GPT-5.2 82.0
  11. 11 Claude Sonnet 4 82.0
  12. 12 qwen3.5-flash-02-23 81.8
  13. 13 Qwen3.5-35B-A3B 81.4
  14. 14 gpt-5-2025-08-07:minimal-reasoning 81.2
  15. 15 GPT-5.4 Mini 81.1
  16. 16 Qwen3.5-122B-A10B 81.0
  17. 17 Gemini 3.1 Flash-Lite 80.9
  18. 18 Claude Haiku 4.5 79.7
  19. 19 GLM-4.7 77.5
  20. 20 GPT-4.1 76.3
  21. 21 Grok 4.1 Fast 75.8
  22. 22 o3 75.1
  23. 23 Qwen3.5-9B 75.0
  24. 24 gpt-5-mini-2025-08-07:minimal-reasoning 74.5
  25. 25 Kimi K2 Jul 2025 73.1
  26. 26 Gemini 2.5 Pro Jun 2025 72.1
  27. 27 Mistral Large 3 68.2
  28. 28 DeepSeek-V3.2 66.3
  29. 29 glm-4.7-flash 66.2
  30. 30 GPT-5.4 Nano 64.5
  31. 31 MiniMax-M2.5 64.5
  32. 32 GLM-4.5 63.4
  33. 33 Gemini 2.5 Flash Sep 2025 63.0
  34. 34 gpt-5-nano-2025-08-07__5x-ensemble 61.1
  35. 35 Qwen3-235B-A22B-Instruct Jul 2025 59.7
  36. 36 Mistral Medium 3 58.5
  37. 37 GPT-4.1 mini 57.3
  38. 38 Qwen3-235B-A22B 56.2
  39. 39 o4-mini 56.1
  40. 40 Gemini 2.5 Flash-Lite Sep 2025 56.0
  41. 41 gpt-oss-120b 54.8
  42. 42 gpt-5-nano-2025-08-07:minimal-reasoning 52.2
  43. 43 Mistral Small 4 51.6
  44. 44 Mistral Small 3.2 50.2
  45. 45 gpt-oss-20b 42.8
  46. 46 Qwen3-30B-A3B-Instruct Jul 2025 34.6
  47. 47 Llama 3.1-8B 19.2
#EntryScoreSettingDateSource
1 Claude Sonnet 4.6 94.1 Feb 2026 Official leaderboard
2 Claude Opus 4.6 92.9 Feb 2026 Official leaderboard
3 Gemini 3 Pro 86.6 Nov 2025 Official leaderboard
4 GLM-5 85.6 Feb 2026 Official leaderboard
5 Qwen3.5 397B-A17B 85.5 Feb 2026 Official leaderboard
6 Gemini 3 Flash 85.4 Dec 2025 Official leaderboard
7 GPT-5.4 85.0 Mar 2026 Official leaderboard
8 Claude Sonnet 4.5 84.6 Sep 2025 Official leaderboard
9 Qwen3.5-27B 84.3 Feb 2026 Official leaderboard
10 GPT-5.2 82.0 Dec 2025 Official leaderboard
11 Claude Sonnet 4 82.0 May 2025 Official leaderboard
12 qwen3.5-flash-02-23 81.8 Official leaderboard
13 Qwen3.5-35B-A3B 81.4 Feb 2026 Official leaderboard
14 gpt-5-2025-08-07:minimal-reasoning 81.2 Official leaderboard
15 GPT-5.4 Mini 81.1 Mar 2026 Official leaderboard
16 Qwen3.5-122B-A10B 81.0 Feb 2026 Official leaderboard
17 Gemini 3.1 Flash-Lite 80.9 Mar 2026 Official leaderboard
18 Claude Haiku 4.5 79.7 Oct 2025 Official leaderboard
19 GLM-4.7 77.5 Dec 2025 Official leaderboard
20 GPT-4.1 76.3 Apr 2025 Official leaderboard
21 Grok 4.1 Fast 75.8 Nov 2025 Official leaderboard
22 o3 75.1 Apr 2025 Official leaderboard
23 Qwen3.5-9B 75.0 Feb 2026 Official leaderboard
24 gpt-5-mini-2025-08-07:minimal-reasoning 74.5 Official leaderboard
25 Kimi K2 Jul 2025 73.1 Jul 2025 Official leaderboard
26 Gemini 2.5 Pro Jun 2025 72.1 Jun 2025 Official leaderboard
27 Mistral Large 3 68.2 Dec 2025 Official leaderboard
28 DeepSeek-V3.2 66.3 Dec 2025 Official leaderboard
29 glm-4.7-flash 66.2 Jan 2026 Official leaderboard
30 GPT-5.4 Nano 64.5 Mar 2026 Official leaderboard
31 MiniMax-M2.5 64.5 Feb 2026 Official leaderboard
32 GLM-4.5 63.4 Aug 2025 Official leaderboard
33 Gemini 2.5 Flash Sep 2025 63.0 Sep 2025 Official leaderboard
34 gpt-5-nano-2025-08-07__5x-ensemble 61.1 Official leaderboard
35 Qwen3-235B-A22B-Instruct Jul 2025 59.7 Jul 2025 Official leaderboard
36 Mistral Medium 3 58.5 May 2025 Official leaderboard
37 GPT-4.1 mini 57.3 Apr 2025 Official leaderboard
38 Qwen3-235B-A22B 56.2 Apr 2025 Official leaderboard
39 o4-mini 56.1 Apr 2025 Official leaderboard
40 Gemini 2.5 Flash-Lite Sep 2025 56.0 Sep 2025 Official leaderboard
41 gpt-oss-120b 54.8 Aug 2025 Official leaderboard
42 gpt-5-nano-2025-08-07:minimal-reasoning 52.2 Official leaderboard
43 Mistral Small 4 51.6 Mar 2026 Official leaderboard
44 Mistral Small 3.2 50.2 Jun 2025 Official leaderboard
45 gpt-oss-20b 42.8 Aug 2025 Official leaderboard
46 Qwen3-30B-A3B-Instruct Jul 2025 34.6 Jul 2025 Official leaderboard
47 Llama 3.1-8B 19.2 Jul 2024 Official leaderboard

7 models have held number one since its release in Jan 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta OpenAI Anthropic Google DeepMind
Own leaderboard Replaced by 4, May 2026
Claude Sonnet 4.6 94.1

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 100.0 2025 2026 0.00 20.0 40.0 60.0 80.0 100.0 2025 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

Replaced by Judgemark v4

More in Writing and taste

All 24 in Writing and taste
  • Design Bench (Moonshot) In use Models make graphic designs from direct prompts or few-shot reference images. An automated grader rates the results as Elo scores. Kimi K3.1 Pro 957
  • WritingBench Fading Models complete 1,000 real-world writing requests across 6 domains and 100 subdomains. A judge model scores each reply on five query-specific criteria. GPT-5 83.9
  • Creative Writing (MiniMax in-house) Only in system cards Models write creative pieces for real Hailuo AI user requests. Automatic assessments score the writing. MiniMax-Text-01 81.3%
  • DeepSeek Chinese functional writing evaluation Only in system cards Models answer 3,170 everyday Chinese writing prompts such as reports, emails and speeches. Pairwise judging against a baseline model gives a win rate. DeepSeek-V4-Pro 62.6%
  • Slide generation human evaluation (Z.ai) Only in system cards People compare HTML slide decks from two models on content, layout and visual quality. Metric is overall win rate. GLM-5 67.5%
  • Task-dependent output homogenization evaluation Only in system cards Models answer prompts several times. A judge scores functional diversity where it is wanted and consistency where one answer is right. Muse Spark 0.21