Benchmark Almanac

tau2-bench

GPT-5.4 98.9% leads since Mar 2026 In use

Dual-control customer-service simulations (retail, airline, telecom) where agent and simulated user both act. Graded on final state.

τ³-bench (v1.0.0, 2026-03) lives in the tau2-bench repo and adds banking_knowledge and voice modes. V1.0.1 fixed banking grading. Telecom scores depend on the user-simulator model.

Made by Sierra Research Released Jun 2025 Customer service All tasks public Last in a launch post Oct 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
sierra-research/tau2-bench 2,366 stars Paper Leaderboard
69
Models scored in lab cards
72
Launch posts list it, from 29 labs
40
System cards report it, from 21 labs

GPT-5.4 leads with 98.9%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

69 models. Scroll sideways for the rest.

  1. 98.9 GPT-5.4
  2. 98.0 GPT-5.5
  3. 97.8 MiniMax-M2.5
  4. 96.8 MiMo-V2-Pro
  5. 95.6 Claude Opus 4.6
  6. 95.3 Ring-2.6-1T
  7. 94.8 Claude Sonnet 4.6
  8. 93.6 Claude Opus 4.5
  9. 92.7 Nova 2.0 Pro Preview
  10. 92.5 GPT-5.4 Nano
  11. 92.3 Seed2.0 Pro
  12. 91.5 Muse Spark
  13. 91.3 Trinity Large Thinking
  14. 90.3 GPT-5.2
  15. 90.2 Gemini 3 Flash
  16. 89.7 GLM-5
  17. 88.2 LongCat-Flash-Thinking-2601
  18. 88.2 Step 3.5 Flash
  19. 87.4 GLM-4.7
  20. 87.0 MiniMax-M2
  21. 87.0 MiniMax-M2.1
  22. 86.9 Spark-X2.6-4B
  23. 86.7 Qwen3.5 397B-A17B
  24. 85.4 Gemini 3 Pro
  25. 85.0 Command A+
  26. 84.7 Claude Sonnet 4.5
  27. 82.6 Keye-VL-2.0-30B-A3B
  28. 82.1 Qwen3-Max
  29. 81.0 Qwen3.5-Omni-Plus
  30. 80.9 Intern-S1-Pro
  31. 80.3 DeepSeek-V3.2
  32. 80.3 MiMo-V2-Flash
  33. 80.2 GPT-5.1
  34. 80.1 GPT-5
  35. 80.0 Nova 2 Omni Preview
  36. 79.5 glm-4.7-flash
  37. 79.5 Qwen3.5-122B-A10B
  38. 78.8 ERNIE 5.0
  39. 78.4 Ring-2.5-1T
  40. 78.4 Ling-2.6-1T
  41. 76.9 Gemma 4 31B IT
  42. 76.6 Claude Haiku 4.5
  43. 76.0 Nova 2 Lite
  44. 75.9 GLM-4.6
  45. 74.0 LongCat-Flash-Thinking
  46. 73.2 K-EXAONE
  47. 73.0 Nemotron 3.5 Super VL
  48. 72.4 Tencent HY 2.0 Think
  49. 72.3 Solar Pro 3 (March 2026 update)
  50. 72.0 Seed1.8
  51. 72.0 EXAONE 4.5 33B
  52. 71.7 MAI-Code-1-Flash
  53. 70.8 GPT-5 mini
  54. 68.3 Sarvam 105B
  55. 68.2 Gemma 4 26B A4B
  56. 67.7 LongCat-Flash-Chat
  57. 66.1 Kimi K2
  58. 64.0 LFM2.5-8B-A1B
  59. 61.1 Nemotron 3 Super
  60. 58.9 Nemotron-Cascade 2 30B-A3B
  61. 58.5 Qwen3-235B-A22B-Thinking
  62. 57.4 Qwen3-Next-80B-A3B
  63. 56.2 DiffusionGemma 26B A4B
  64. 55.8 Solar Open 100B
  65. 52.3 Nemotron 3.5 Bolt E4B
  66. 49.0 Nemotron 3 Nano 30B-A3B
  67. 46.3 GPT-5 nano
  68. 45.7 Sarvam 30B
  69. 35.4 Nemotron 3 Nano 4B
  1. 1 GPT-5.4 98.9%
  2. 2 GPT-5.5 98%
  3. 3 MiniMax-M2.5 97.8%
  4. 4 MiMo-V2-Pro 96.8%
  5. 5 Claude Opus 4.6 95.6%
  6. 6 Ring-2.6-1T 95.3%
  7. 7 Claude Sonnet 4.6 94.8%
  8. 8 Claude Opus 4.5 93.6%
  9. 9 Nova 2.0 Pro Preview 92.7%
  10. 10 GPT-5.4 Nano 92.5%
  11. 11 Seed2.0 Pro 92.3%
  12. 12 Muse Spark 91.5%
  13. 13 Trinity Large Thinking 91.3%
  14. 14 GPT-5.2 90.3%
  15. 15 Gemini 3 Flash 90.2%
  16. 16 GLM-5 89.7%
  17. 17 LongCat-Flash-Thinking-2601 88.2%
  18. 18 Step 3.5 Flash 88.2%
  19. 19 GLM-4.7 87.4%
  20. 20 MiniMax-M2 87%
  21. 21 MiniMax-M2.1 87%
  22. 22 Spark-X2.6-4B 86.9%
  23. 23 Qwen3.5 397B-A17B 86.7%
  24. 24 Gemini 3 Pro 85.4%
  25. 25 Command A+ 85%
  26. 26 Claude Sonnet 4.5 84.7%
  27. 27 Keye-VL-2.0-30B-A3B 82.6%
  28. 28 Qwen3-Max 82.1%
  29. 29 Qwen3.5-Omni-Plus 81%
  30. 30 Intern-S1-Pro 80.9%
  31. 31 DeepSeek-V3.2 80.3%
  32. 32 MiMo-V2-Flash 80.3%
  33. 33 GPT-5.1 80.2%
  34. 34 GPT-5 80.1%
  35. 35 Nova 2 Omni Preview 80%
  36. 36 glm-4.7-flash 79.5%
  37. 37 Qwen3.5-122B-A10B 79.5%
  38. 38 ERNIE 5.0 78.8%
  39. 39 Ring-2.5-1T 78.4%
  40. 40 Ling-2.6-1T 78.4%
  41. 41 Gemma 4 31B IT 76.9%
  42. 42 Claude Haiku 4.5 76.6%
  43. 43 Nova 2 Lite 76%
  44. 44 GLM-4.6 75.9%
  45. 45 LongCat-Flash-Thinking 74%
  46. 46 K-EXAONE 73.2%
  47. 47 Nemotron 3.5 Super VL 73%
  48. 48 Tencent HY 2.0 Think 72.4%
  49. 49 Solar Pro 3 (March 2026 update) 72.3%
  50. 50 Seed1.8 72%
  51. 51 EXAONE 4.5 33B 72%
  52. 52 MAI-Code-1-Flash 71.7%
  53. 53 GPT-5 mini 70.8%
  54. 54 Sarvam 105B 68.3%
  55. 55 Gemma 4 26B A4B 68.2%
  56. 56 LongCat-Flash-Chat 67.7%
  57. 57 Kimi K2 Jul 2025 66.1%
  58. 58 LFM2.5-8B-A1B 64%
  59. 59 Nemotron 3 Super 61.1%
  60. 60 Nemotron-Cascade 2 30B-A3B 58.9%
  61. 61 Qwen3-235B-A22B-Thinking Jul 2025 58.5%
  62. 62 Qwen3-Next-80B-A3B 57.4%
  63. 63 DiffusionGemma 26B A4B 56.2%
  64. 64 Solar Open 100B 55.8%
  65. 65 Nemotron 3.5 Bolt E4B 52.3%
  66. 66 Nemotron 3 Nano 30B-A3B 49%
  67. 67 GPT-5 nano 46.3%
  68. 68 Sarvam 30B 45.7%
  69. 69 Nemotron 3 Nano 4B 35.4%
#ModelScoreSettingDateSource
1 GPT-5.4 98.9% Mar 2026 Model card
2 GPT-5.5 98% Apr 2026 Model card
3 MiniMax-M2.5 97.8% Feb 2026 Model card
4 MiMo-V2-Pro 96.8% Mar 2026 Model card
5 Claude Opus 4.6 95.6% Feb 2026 Model card
6 Ring-2.6-1T 95.3% May 2026 Model card
7 Claude Sonnet 4.6 94.8% Feb 2026 Model card
8 Claude Opus 4.5 93.6% Nov 2025 Model card
9 Nova 2.0 Pro Preview 92.7% Dec 2025 Model card
10 GPT-5.4 Nano 92.5% Mar 2026 Model card
11 Seed2.0 Pro 92.3% Feb 2026 Model card
12 Muse Spark 91.5% Apr 2026 Model card
13 Trinity Large Thinking 91.3% Apr 2026 Model card
14 GPT-5.2 90.3% Dec 2025 Model card
15 Gemini 3 Flash 90.2% Dec 2025 Model card
16 GLM-5 89.7% Feb 2026 Model card
17 LongCat-Flash-Thinking-2601 88.2% Jan 2026 Model card
18 Step 3.5 Flash 88.2% Feb 2026 Model card
19 GLM-4.7 87.4% Dec 2025 Model card
20 MiniMax-M2 87% Oct 2025 Model card
21 MiniMax-M2.1 87% Dec 2025 Model card
22 Spark-X2.6-4B 86.9% Oct 2026 Model card
23 Qwen3.5 397B-A17B 86.7% Feb 2026 Model card
24 Gemini 3 Pro 85.4% Nov 2025 Model card
25 Command A+ 85% May 2026 Model card
26 Claude Sonnet 4.5 84.7% Sep 2025 Model card
27 Keye-VL-2.0-30B-A3B 82.6% May 2026 Model card
28 Qwen3-Max 82.1% Jan 2026 Model card
29 Qwen3.5-Omni-Plus 81% Mar 2026 Model card
30 Intern-S1-Pro 80.9% Feb 2026 Model card
31 DeepSeek-V3.2 80.3% Dec 2025 Model card
32 MiMo-V2-Flash 80.3% Dec 2025 Model card
33 GPT-5.1 80.2% Nov 2025 Model card
34 GPT-5 80.1% Aug 2025 Model card
35 Nova 2 Omni Preview 80% Dec 2025 Model card
36 glm-4.7-flash 79.5% Jan 2026 Model card
37 Qwen3.5-122B-A10B 79.5% Feb 2026 Model card
38 ERNIE 5.0 78.8% Feb 2026 Model card
39 Ring-2.5-1T 78.4% Feb 2026 Model card
40 Ling-2.6-1T 78.4% Apr 2026 Model card
41 Gemma 4 31B IT 76.9% Apr 2026 Model card
42 Claude Haiku 4.5 76.6% Oct 2025 Model card
43 Nova 2 Lite 76% Dec 2025 Model card
44 GLM-4.6 75.9% Sep 2025 Model card
45 LongCat-Flash-Thinking 74% Sep 2025 Model card
46 K-EXAONE 73.2% Jan 2026 Model card
47 Nemotron 3.5 Super VL 73% Oct 2026 Model card
48 Tencent HY 2.0 Think 72.4% Dec 2025 Model card
49 Solar Pro 3 (March 2026 update) 72.3% Mar 2026 Model card
50 Seed1.8 72% Dec 2025 Model card
51 EXAONE 4.5 33B 72% Apr 2026 Model card
52 MAI-Code-1-Flash 71.7% Jun 2026 Model card
53 GPT-5 mini 70.8% Aug 2025 Model card
54 Sarvam 105B 68.3% Mar 2026 Model card
55 Gemma 4 26B A4B 68.2% Apr 2026 Model card
56 LongCat-Flash-Chat 67.7% Aug 2025 Model card
57 Kimi K2 Jul 2025 66.1% Jul 2025 Model card
58 LFM2.5-8B-A1B 64% May 2026 Model card
59 Nemotron 3 Super 61.1% Mar 2026 Model card
60 Nemotron-Cascade 2 30B-A3B 58.9% Mar 2026 Model card
61 Qwen3-235B-A22B-Thinking Jul 2025 58.5% Jul 2025 Model card
62 Qwen3-Next-80B-A3B 57.4% Sep 2025 Model card
63 DiffusionGemma 26B A4B 56.2% Jun 2026 Model card
64 Solar Open 100B 55.8% Dec 2025 Model card
65 Nemotron 3.5 Bolt E4B 52.3% Oct 2026 Model card
66 Nemotron 3 Nano 30B-A3B 49% Dec 2025 Model card
67 GPT-5 nano 46.3% Aug 2025 Model card
68 Sarvam 30B 45.7% Mar 2026 Model card
69 Nemotron 3 Nano 4B 35.4% Mar 2026 Model card

The benchmark's own leaderboard has 10 models. We plot every source in the charts below.

6 models have held number one since its release in Jun 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Anthropic Moonshot AI OpenAI Qwen Google DeepMind MiniMax
Own leaderboard Passed 85% in Nov 2025, by Gemini 3 Pro
Gemini 3 Pro 85.4%
Lab claims Passed 85% in Oct 2025, by MiniMax-M2
GPT-5.4 98.9%

The best score passed 85% in Oct 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-5.4 Claude Opus 4.5 MiniMax-M2 Qwen3-Max Kimi K2 Claude 3.7 Sonnet 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-5.4 MiniMax-M2

4 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Moonshot AI
Qwen
OpenAI
Meituan
Anthropic
Z.ai
MiniMax
Google DeepMind
xAI
DeepSeek
Amazon
Tencent
NVIDIA
Xiaomi Corp
ByteDance Seed
LG AI Research
Upstage
StepFun
Shanghai AI Lab
Baidu
Ant Group
Sarvam AI
Arcee AI
Meta
Cohere
Kuaishou
Liquid AI
Microsoft
iFlytek
DateLaunch postScoreAs the post writes it
23 Oct 2026 Nemotron 3.5 Bolt E4B 52.3 Tau2-Bench
22 Oct 2026 Spark-X2.6-4B 86.9 τ²-bench
15 Oct 2026 Nemotron 3.5 Super VL 73 Mean of tau2-bench Retail 69.24 / Airline 76.50 / Telecom 73.19
10 Jun 2026 DiffusionGemma 26B A4B 56.2 Tau2 (average over 3)
2 Jun 2026 MAI-Code-1-Flash 71.7 𝜏²-Bench telecom
28 May 2026 LFM2.5-8B-A1B 64 Mean of Tau² Telecom 88.07 / Tau² Retail 39.82
25 May 2026 Keye-VL-2.0-30B-A3B 82.6 TAU2-Bench
20 May 2026 Command A+ 85 τ²-Bench Telecom only (chart label)
14 May 2026 Ring-2.6-1T 95.3 Tau2-Bench Telecom scenario only (high)
29 Apr 2026 Ling-2.6-1T 78.4 TAU2-Bench
23 Apr 2026 GPT-5.5 98 Tau2-bench Telecom (original prompts)
9 Apr 2026 EXAONE 4.5 33B 69.1 Mean of τ2-Bench Retail 77.9 / Airline 56.5 / Telecom 73.0
8 Apr 2026 Muse Spark 91.5 τ²-Bench Telecom (Agentic Tool Use (Artificial Analysis))
2 Apr 2026 Gemma 4 26B A4B 68.2 Tau2 (average over 3)
2 Apr 2026 Gemma 4 31B IT 76.9 Tau2 (average over 3)
1 Apr 2026 Trinity Large Thinking 91.3 Mean of Tau2-Airline 88.0 / Tau2-Telecom 94.7
30 Mar 2026 Qwen3.5-Omni-Plus 81 TAU2Bench
24 Mar 2026 Solar Pro 3 (March 2026 update) 72.3 Tau2-all
19 Mar 2026 Nemotron-Cascade 2 30B-A3B 58.9 τ²-Bench
18 Mar 2026 MiMo-V2-Pro 96.8 τ2-bench (Telecom)
17 Mar 2026 GPT-5.4 Nano 92.5 τ2-bench (telecom)
16 Mar 2026 Nemotron 3 Nano 4B 35.4 Mean of Tau2 Airline 33.3 / Retail 39.8 / Telecom 33 (reasoning on)
11 Mar 2026 Nemotron 3 Super 61.1 TauBench V2 Average of Airline 56.25 / Retail 62.83 / Telecom 64.36
6 Mar 2026 Sarvam 30B 45.7 Tau2 (avg.)
6 Mar 2026 Sarvam 105B 68.3 Tau2 (avg.)
5 Mar 2026 GPT-5.4 98.9 Tau2-bench Telecom (xhigh; body chart shows 64.3% without reasoning)
24 Feb 2026 Qwen3.5-122B-A10B 79.5 TAU2-Bench (airline fixes from Claude Opus 4.5 system card)
19 Feb 2026 Gemini 3.1 Pro Mean of τ2-bench, Retail / τ2-bench, Telecom
17 Feb 2026 Claude Sonnet 4.6 94.8 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
16 Feb 2026 Qwen3.5 397B-A17B 86.7 TAU2-Bench
14 Feb 2026 Seed2.0 Pro 92.3 Mean of τ2-Bench (retail) 90.4 / τ2-Bench (telecom) 94.2
12 Feb 2026 MiniMax-M2.5 97.8 τ² Telecom
11 Feb 2026 GLM-5 89.7 τ²-Bench
10 Feb 2026 Ring-2.5-1T 78.4 Tau2-Bench (Mean@3)
6 Feb 2026 ERNIE 5.0 78.8 TAU2-Bench
5 Feb 2026 Claude Opus 4.6 95.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
2 Feb 2026 Intern-S1-Pro 80.9 Tau2-Bench
2 Feb 2026 Step 3.5 Flash 88.2 τ²-Bench
23 Jan 2026 Qwen3-Max 82.1 Tau² Bench
19 Jan 2026 glm-4.7-flash 79.5 τ²-Bench
14 Jan 2026 LongCat-Flash-Thinking-2601 88.2 τ²-Avg (Avg@4): Retail 88.6 / Airline 76.5 / Telecom 99.3
31 Dec 2025 Solar Open 100B 55.8 Mean of Tau² Airline 52.4 / Telecom 55.6 / Retail 59.3
31 Dec 2025 K-EXAONE 70.8 Mean of τ2-Bench Retail 78.6 / Airline 60.4 / Telecom 73.5
23 Dec 2025 MiniMax-M2.1 87 𝜏²-Bench Telecom
22 Dec 2025 GLM-4.7 87.4 τ²-Bench
18 Dec 2025 Seed1.8 72 τ2-Bench
17 Dec 2025 Gemini 3 Flash 90.2 τ2-bench
16 Dec 2025 MiMo-V2-Flash 80.3 τ²-Bench
15 Dec 2025 Nemotron 3 Nano 30B-A3B 49 TauBench V2 (Average) of Airline 48.0 / Retail 56.9 / Telecom 42.2
11 Dec 2025 GPT-5.2 90.3 Mean of Tau2-bench Telecom Tool use in customer support / Tau2-bench Retail Tool use in customer support
5 Dec 2025 Tencent HY 2.0 Think 72.4 Tau2-Bench
2 Dec 2025 Nova 2 Omni Preview 80 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
2 Dec 2025 Nova 2 Lite 76 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
2 Dec 2025 Nova 2.0 Pro Preview 92.7 τ2-bench Telecom only (as benchmarked by Artificial Analysis)
1 Dec 2025 DeepSeek-V3.2 80.3 τ2-Bench (Pass@1)
24 Nov 2025 Claude Opus 4.5 93.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Telecom
19 Nov 2025 Grok 4.1 Fast τ²-bench Telecom (chart)
18 Nov 2025 Gemini 3 Pro 85.4 τ2-bench
13 Nov 2025 GPT-5.1 80.2 Mean of Tau2-bench Airline / Tau2-bench Telecom* / Tau2-bench Retail
27 Oct 2025 MiniMax-M2 87 Best of τ²-Bench / τ²-Bench-Telecom (AA)
15 Oct 2025 Claude Haiku 4.5 76.6 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Airline / Agentic tool use τ2-bench Telecom
30 Sep 2025 GLM-4.6 75.9 τ²-Bench (Weighted)
29 Sep 2025 Claude Sonnet 4.5 84.7 Mean of Agentic tool use τ2-bench Retail / Agentic tool use τ2-bench Airline / Agentic tool use τ2-bench Telecom
24 Sep 2025 Qwen3-Max 74.8 Tau2-Bench (Weighted)
21 Sep 2025 LongCat-Flash-Thinking 74 Mean of τ²-Bench-Retail 71.5 / τ²-Bench-Airline 67.5 / τ²-Bench-Telecom 83.1 (Mean@4)
11 Sep 2025 Qwen3-Next-80B-A3B 57.4 Mean of TAU2-Retail 67.8 / TAU2-Airline 60.5 / TAU2-Telecom 43.9
29 Aug 2025 LongCat-Flash-Chat 67.7 Mean of τ²-Bench (telecom) 73.68 / (airline) 58.00 / (retail) 71.27 (avg@4)
7 Aug 2025 GPT-5 nano 46.3 Mean of Tau2-bench airline 41.0 / retail 62.3 / telecom 35.5
7 Aug 2025 GPT-5 mini 70.8 Mean of Tau2-bench airline 60.0 / retail 78.3 / telecom 74.1
7 Aug 2025 GPT-5 80.1 Mean of Tau2-bench Function calling (airline) / Tau2-bench Function calling (retail) / Tau2-bench Function calling (telecom)
25 Jul 2025 Qwen3-235B-A22B-Thinking 58.5 Mean of TAU2-Retail / TAU2-Airline / TAU2-Telecom
11 Jul 2025 Kimi K2 64.3 Mean of Tau2 retail, Avg@4 / Tau2 airline, Avg@4 / Tau2 telecom, Avg@4

40 system cards report it, from 21 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
23 Oct 2026 Nemotron 3.5 Bolt E4B technical report 52.3 tau2-Bench, released Q4_K_M imatrix checkpoint (Table 5). BF16 55.1 (Table 3) Capabilities
19 Oct 2026 Muse Glimmer technical report TauBench, three splits with no overall, high: Airline 78.0, Retail 87.1, Telecom 94.1. The telecom split suggests tau2-bench, though the text cites Yao et al. 2024 (Table 5) Capabilities
15 Oct 2026 Nemotron 3.5 Super VL technical report Tau2 Retail 69.24, Airline 76.50, Telecom 73.19, no overall score (Table 10) Capabilities
15 Oct 2026 Nemotron 3.5 Lightning 30B-A3B technical report Tau2 Telecom only, BF16 acc. 59.65 (Table 22) Capabilities
31 Jul 2026 DiffusionGemma 26B A4B technical report Tau2, three domains with no overall score: Retail 71.5, Airline 69.0, Telecom 28.1, TD (Table 3) Capabilities
24 Jul 2026 Gemma 4 31B IT technical report Tau2, three domains with no overall score: airline 75.0, retail 86.4, telecom 69.3 (Table 5) Capabilities
24 Jul 2026 Gemma 4 26B A4B technical report Tau2, three domains with no overall score: airline 76.0, retail 85.5, telecom 43.0 (Table 5) Capabilities
13 Jun 2026 Ring-2.6-1T technical report 83.4 τ2-Average (Acc), xhigh, Retail 77.85, Airline 77.50, Telecom 94.96 (Table 6). High gives Average 84.26, Telecom 96.71 Capabilities
13 Jun 2026 Ling-2.6-1T technical report 78.4 τ2-bench, Mean@4, user model gpt-5.2, average of Retail, Airline and Telecom (Table 5) Capabilities
9 Jun 2026 Keye-VL-2.0-30B-A3B technical report 82.6 τ2-Bench (Table 4) Capabilities
2 Jun 2026 MAI-Code-1-Flash model card 71.7 tau2-Bench telecom, agentic tool use (Instruction following table) Capabilities
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report TauBench V2 (Telecom) only, 42.7, no overall score (Table 10) Capabilities
17 Apr 2026 Qwen3.5-Omni-Plus technical report 81 TAU2Bench (Table 4) Capabilities
9 Apr 2026 EXAONE 4.5 33B technical report 72 tau2-Bench weighted average stated in the 3.3 text. Retail 77.9, Airline 56.5, Telecom 73.0 (Table 3, Reasoning mode) Capabilities
26 Mar 2026 Intern-S1-Pro technical report 80.9 τ2-Bench, official settings, thinking (Table 2) Capabilities
21 Mar 2026 Seed1.8 model card 72 τ2-Bench (Table 4) Capabilities
19 Mar 2026 Nemotron-Cascade 2 30B-A3B technical report 58.9 τ2-Bench, latest-turn thought retention, which the report says adds 3 to 5 points over the official policy (Table 1 and A.6) Capabilities
11 Mar 2026 Nemotron 3 Super technical report 61.1 TauBench V2 Average of Airline 56.25, Retail 62.83, Telecom 64.36 (Table 5) Capabilities
19 Feb 2026 Gemini 3.1 Pro model card τ2-bench, Retail 90.8% and Telecom 99.3%, no overall (results table, p.4) Capabilities
17 Feb 2026 GLM-5 technical report 89.7 τ2-Bench, prompt-adjusted user simulator (Table 7) Capabilities
17 Feb 2026 Claude Sonnet 4.6 system card τ²-bench, Retail 91.7 and Telecom 97.9, Airline not reported, no overall (Table 2.1.A, 2.5) Capabilities
14 Feb 2026 Seed2.0 Pro model card τ2-Bench retail 90.4, telecom 94.2, no overall (Table 11) Capabilities
11 Feb 2026 Step 3.5 Flash technical report 88.2 τ2-Bench, avg of 8 runs, fixed airline tasks (Table 5 and E.2.3) Capabilities
5 Feb 2026 Claude Opus 4.6 system card τ²-bench, Retail 91.9 and Telecom 99.3, Airline not reported, no overall (Table 2.3.A, 2.7) Capabilities
4 Feb 2026 ERNIE 5.0 technical report 78.8 TAU2-Bench (Table 2) Capabilities
23 Jan 2026 LongCat-Flash-Thinking-2601 technical report 88.2 τ2-Avg (Avg@4), GPT-4.1 user simulator, cleaned airline subset, Retail 88.6, Airline 76.5, Telecom 99.3 (Table 2) Capabilities
11 Jan 2026 Solar Open 100B technical report Tau2 Airline 52.4, Telecom 55.6, Retail 59.3, no overall score (Table 6) Capabilities
6 Jan 2026 MiMo-V2-Flash technical report 80.3 τ2-Bench, DeepSeek-V3.2 as user agent (Table 9). Telecom 95.3, Retail 79.5, Airline 66.0 (4.5.2) Capabilities
5 Jan 2026 Falcon H1R 7B technical report τ2-Telecom only, 25.4, no overall tau2-bench score (Table 5) Capabilities
5 Jan 2026 K-EXAONE technical report 73.2 tau2-Bench weighted average stated in the 4.2 text. Retail 78.6, Airline 60.4, Telecom 73.5 (Table 3, Reasoning mode). Non-reasoning 73.2, 42.6, 44.0 (Table 4) Capabilities
23 Dec 2025 Nemotron 3 Nano 30B-A3B technical report 49 TauBench V2 Average (Table 3). Airline 48.00, Retail 56.91, Telecom 42.21 Capabilities
2 Dec 2025 Nova 2 Omni Preview technical report tau2-bench Telecom only 80.0, numbers as benchmarked by Artificial Analysis (Table 5) Capabilities
2 Dec 2025 Nova 2 Lite technical report tau2-bench Telecom only 76.0, numbers as benchmarked by Artificial Analysis (Table 2) Capabilities
2 Dec 2025 Nova 2.0 Pro Preview technical report tau2-bench Telecom only 92.7, numbers as benchmarked by Artificial Analysis (Table 2) Capabilities
1 Dec 2025 DeepSeek-V3.2 technical report 80.3 τ2-Bench (Pass@1), average of categories: Airline 63.8, Retail 81.1, Telecom 96.2, DeepSeek-V3.2 Thinking (Table 2) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report TAU2-Retail 67.0, TAU2-Airline 62.0, TAU2-Telecom 44.7, no overall (Table 6) Capabilities
24 Nov 2025 Claude Opus 4.5 system card τ²-Bench, Retail 88.9% and Telecom 98.2% without extended thinking, no overall (Table 2.3.A). Airline 67.9% original and 87.8% corrected per Table 2.3.A footnote 6. Table 2.8.A prints Airline original 70.1% Capabilities
23 Sep 2025 LongCat-Flash-Thinking technical report τ2-Bench-Retail 71.5, Airline 67.5, Telecom 83.1 (Mean@4), no overall score (Table 2) Capabilities
1 Sep 2025 LongCat-Flash-Chat technical report τ2-Bench telecom 73.68, airline 58.00, retail 71.27 (avg@4), no overall score (Table 3) Capabilities
21 Jul 2025 Kimi K2 technical report 66.1 τ2-Bench micro Pass@1 across domains (Appendix C text). Avg@4: retail 70.6, airline 56.5, telecom 65.8 (Table 3) Capabilities

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI Moonshot AI Qwen MiniMax Inception Labs
tau3-bench Passed 85% in Mar 2026, by Qwen3.5 397B-A17B
Mercury 2.5 96%
tau-bench Replaced by tau2-bench, Sep 2025
Claude Opus 4 70.5%
tau2-bench Passed 85% in Oct 2025, by MiniMax-M2
GPT-5.4 98.9%
tau3-bench Passed 85% in Mar 2026, by Qwen3.5 397B-A17B
Mercury 2.5 96%
Replaces tau-bench Replaced by tau3-bench

More in Tool use

All 125 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Muse Spark 1.1 88.1%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Claw-Eval In use 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. Claude Sonnet 4.6 81.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%