Benchmark Almanac

Thrive Holdings helpdesk ticket resolution benchmark

GPT-6 Astra 61.5% leads since Oct 2026 Not in model cards

Agent investigates IT support requests, takes action and decides when to escalate to a technician. Scored by pass rate.

Item count not stated. Drawn from Thrive's IT services firms.

Made by Thrive Holdings Released Oct 2026 Tasks from real jobs Tasks kept private Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
University Undergraduates and entry-level professionals
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
Leaderboard
15
Models scored on its own leaderboard

GPT-6 Astra leads with 61.5%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

15 models. Scroll sideways for the rest.

  1. 61.5 GPT-6 Astra
  2. 59.6 Claude Opus 5.5
  3. 55.8 Claude Fable 5.1
  4. 55.8 GPT-6 Sol
  5. 51.9 Muse Spark 1.3
  6. 51.9 Claude Sonnet 5.5
  7. 48.1 Kimi K3
  8. 48.1 GLM-5.3
  9. 48.1 DeepSeek V4.1 Flash
  10. 46.1 Grok 4.7
  11. 42.3 GLM-5.3-Flash
  12. 36.5 Gemini 3.8 Flash
  13. 34.6 GPT-6 Luna
  14. 32.7 DeepSeek V4 Pro 0813
  15. 23.1 MiniMax-M3
  1. 1 GPT-6 Astra xhigh 61.5%
  2. 2 Claude Opus 5.5 max 59.6%
  3. 3 Claude Fable 5.1 xhigh 55.8%
  4. 4 GPT-6 Sol max 55.8%
  5. 5 Muse Spark 1.3 xhigh 51.9%
  6. 6 Claude Sonnet 5.5 xhigh 51.9%
  7. 7 Kimi K3 low 48.1%
  8. 8 GLM-5.3 max 48.1%
  9. 9 DeepSeek V4.1 Flash low 48.1%
  10. 10 Grok 4.7 high 46.1%
  11. 11 GLM-5.3-Flash max 42.3%
  12. 12 Gemini 3.8 Flash high 36.5%
  13. 13 GPT-6 Luna xhigh 34.6%
  14. 14 DeepSeek V4 Pro 0813 xhigh 32.7%
  15. 15 MiniMax-M3 high 23.1%
#ModelScoreSettingDateSource
1 GPT-6 Astra 61.5% xhigh Sep 2026 Official leaderboard
2 Claude Opus 5.5 59.6% max Sep 2026 Official leaderboard
3 Claude Fable 5.1 55.8% xhigh Sep 2026 Official leaderboard
4 GPT-6 Sol 55.8% max Sep 2026 Official leaderboard
5 Muse Spark 1.3 51.9% xhigh Sep 2026 Official leaderboard
6 Claude Sonnet 5.5 51.9% xhigh Sep 2026 Official leaderboard
7 Kimi K3 48.1% low Jul 2026 Official leaderboard
8 GLM-5.3 48.1% max Aug 2026 Official leaderboard
9 DeepSeek V4.1 Flash 48.1% low Sep 2026 Official leaderboard
10 Grok 4.7 46.1% high Sep 2026 Official leaderboard
11 GLM-5.3-Flash 42.3% max Aug 2026 Official leaderboard
12 Gemini 3.8 Flash 36.5% high Sep 2026 Official leaderboard
13 GPT-6 Luna 34.6% xhigh Sep 2026 Official leaderboard
14 DeepSeek V4 Pro 0813 32.7% xhigh Aug 2026 Official leaderboard
15 MiniMax-M3 23.1% high Jun 2026 Official leaderboard

The best score, 61.5%, costs $7.63 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Meta OpenAI DeepSeek Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, xhigh Claude Opus 5.5, xhigh Muse Spark 1.3, xhigh DeepSeek V4.1 Flash, low GPT-6 Luna, xhigh GPT-6 Luna, high GPT-6 Luna, medium 0% 20% 40% 60% 80% $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, xhigh GPT-6 Sol, max
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra xhigh 61.5% $7.63
GPT-6 Astra high 59.6% $7.09
Claude Opus 5.5 max 59.6% $8.25
GPT-6 Astra low 57.7% $5.12
Claude Opus 5.5 xhigh 57.7% $4.13
Claude Fable 5.1 xhigh 55.8% $14
GPT-6 Sol max 55.8% $2.25
Claude Fable 5.1 high 53.9% $11
GPT-6 Astra medium 53.9% $6.04
GPT-6 Astra max 53.9% $8.36
Claude Opus 5.5 high 53.9% $3.26
Muse Spark 1.3 xhigh 51.9% $1.42
GPT-6 Sol xhigh 51.9% $1.77
Claude Sonnet 5.5 xhigh 51.9% $2.46
Claude Fable 5.1 max 50% $18
Claude Opus 5.5 medium 50% $2.89
GPT-6 Sol high 50% $1.45
Kimi K3 low 48.1% $1.79
GLM-5.3 max 48.1% $1.48
Claude Fable 5.1 medium 48.1% $9.80
DeepSeek V4.1 Flash low 48.1% $0.13
Claude Sonnet 5.5 high 48.1% $1.50
Claude Fable 5.1 low 46.1% $9.37
Grok 4.7 high 46.1% $3.51
GPT-6 Sol low 46.1% $0.97
GPT-6 Sol medium 46.1% $1.18
Claude Sonnet 5.5 max 46.1% $6.20
Kimi K3 high 44.2% $1.65
Kimi K3 max 42.3% $1.89
GLM-5.3-Flash max 42.3% $0.20
Muse Spark 1.3 high 42.3% $2.03
DeepSeek V4.1 Flash max 42.3% $0.20
Grok 4.7 medium 42.3% $3.06
Muse Spark 1.3 medium 40.4% $1.89
GLM-5.3 high 38.5% $1.14
DeepSeek V4.1 Flash high 38.5% $0.14
Claude Opus 5.5 low 38.5% $2.33
Gemini 3.8 Flash high 36.5% $1.59
Muse Spark 1.3 low 36.5% $1.74
Muse Spark 1.3 max 36.5% $2.47
Gemini 3.8 Flash medium 34.6% $1.15
Grok 4.7 low 34.6% $2.28
GPT-6 Luna xhigh 34.6% $0.09
Claude Sonnet 5.5 medium 34.6% $1.23
DeepSeek V4 Pro 0813 xhigh 32.7% $0.53
GLM-5.3 low 32.7% $0.80
GLM-5.3-Flash high 32.7% $0.16
Gemini 3.8 Flash low 30.8% $0.84
GPT-6 Luna high 30.8% $0.08
GPT-6 Luna max 30.8% $0.13
Claude Sonnet 5.5 low 30.8% $1.23
DeepSeek V4 Pro 0813 high 28.9% $0.53
Grok 4.7 xhigh 28.9% $4.31
GLM-5.3-Flash low 26.9% $0.13
GPT-6 Luna medium 26.9% $0.07
MiniMax-M3 high 23.1% $0.27
GPT-6 Luna low 19.2% $0.05

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-6 Astra 61.5%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6 Astra Claude Fable 5.1 Kimi K3 MiniMax-M3 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6 Astra Kimi K3

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Economic work

All 91 in Economic work
  • JobBench In use 130 tasks from 35 occupations workers most want delegated. Agent works through messy reference files, graded by fact-anchored binary rubrics. GLM-5.3 61.4%
  • GDPval In use Real work tasks from 44 occupations with reference files. Experts blindly compare model deliverables with expert work. Metric is win-or-tie rate. Seed2.1 Pro 87.9%
  • GDPval-AA v2.1 In use Agents with shell and web access complete 220 GDPval gold tasks from 44 occupations. Blind pairwise comparisons give Elo ratings. Claude Opus 5.5 1,846
  • AA-Briefcase v1.1 In use Version 1.1 of Artificial Analysis's multi-hour office work evaluation, with Elo ratings fitted by Crowd-BT. Claude Opus 5.5 1,822
  • CoWorkBench In use Agent completes long-horizon productivity tasks across computer science, finance, law and medicine. Qwen in-house scoring (method unpublished). Qwen3.8-Omni-Flash 75.3%
  • OfficeQA In use Grounded reasoning over a large corpus of U.S. Treasury Bulletin documents: locate tables and compute precise numeric answers. Claude Opus 4.7 86.3%