Terminal agents
We track 18 benchmarks in terminal agents, or 23 if you count each version. They run from everyday level to research frontier level.
OpenAI held the top terminal agents score 38% of the contested time
That's about 18 of the 46 months since January 2023, averaged over the benchmarks below. The easiest benchmarks are at the top. Versions of one test share a row, with a dark tick at each new version. We stop the clock, and fade the rest of the row, when the best score passes 85%, or at the newest score if 45 days go by without another.
Mistral AI OpenAI Google DeepMind Anthropic Cognition MiniMax Xiaomi Corp Z.ai Qwen xAI DeepSeek Tencent
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Share of contested time at number one
OpenAI 38% Anthropic 24% Z.ai 13% 
Google DeepMind 10% 
Qwen 6% 
DeepSeek 4% 
MiniMax 3% 
Mistral AI 2%
Google DeepMind 10% Qwen 6% DeepSeek 4% MiniMax 3% Mistral AI 2% Scores come from each benchmark’s own leaderboard, from Epoch AI’s runs and from labs’ reports on their own models. We date Epoch runs by the model’s release and lab reports by when they came out. Labs pick their own settings, so a lab’s claim can beat an independent run of the same model. Turn off lab claims to see independent runs only.
Mistral AI OpenAI Google DeepMind Anthropic xAI DeepSeek
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Share of contested time at number one, independent runs only
OpenAI 51% Google DeepMind 20% Anthropic 20% DeepSeek 6% Mistral AI 3%
Google DeepMind 20% Anthropic 20% DeepSeek 6% Mistral AI 3% Only each benchmark’s own leaderboard and Epoch AI’s runs, dated by the model’s release. We leave out benchmarks that only labs report.
From everyday to research frontier level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Expert
Specialists with years of training, often over hours of work
- Terminal-Bench 2.0
- Terminal-Bench 2.1
- Terminal-Bench 4.0
- Terminal-Bench
- SkillsBench
- Long-Horizon-Terminal-Bench (LHTB)
- CompileBench
- Terminal-Bench Hard
- Terminal-Bench 3.0
- Long-Horizon-Terminal-Bench 2.0
- ITBench
- DeepTerminalBench
- o11y-bench
- EdgeBench
- aws-bench
- ITBench-AA
- OpenThoughts-TBLite
- Terminal-Bench 2.0 Verified (Z.ai)
- Terminal-Bench 2.1 Extended
- Terminal-X
Research frontier
Researchers, on open problems or tasks that take days
By kind
Terminal tasks
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Terminal-Bench 2.0 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Expert | Checked automatically | 84.7% | In use |
| Terminal-Bench 2.1 89 hard terminal tasks across 16 categories in containers. Hidden tests verify the final state. 5 trials per task. | Expert | Checked automatically | 87.4% | In use |
| Terminal-Bench 4.0 Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. | Expert | Checked automatically | 64.8% | In use |
| Terminal-Bench Agents complete about 80 hard tasks in sandboxed Linux terminals, such as compiling code or configuring servers. Test scripts check the final container state. | Expert | Checked automatically | 49.6% | Fading |
| SkillsBench 86-87 containerized expert tasks in 11 domains, run with and without Agent Skills. Verifier tests give pass rate and skill lift. | Expert | Checked automatically | 67.3% | In use |
| CompileBench Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. | Expert | Checked automatically | 100% | Not in cards |
| Terminal-Bench Hard Agents complete 44 hard tasks from Terminal-Bench Core in the Terminus 2 harness. Artificial Analysis scores pass@1 averaged over 3 repeats. | Expert | Checked automatically | 36% | In use |
| Terminal-Bench 3.0 Agents complete 74 tasks across 7 domains in sandboxed terminal environments. Verifier scripts grade each trial as pass or fail. | Expert | Checked automatically | 42.7% | In use |
| ITBench Agents handle IT automation tasks in Kubernetes environments: site reliability incidents, compliance and security checks, and cloud cost work. | Expert | Checked automatically | 58.3% | Not in cards |
| Harbor-Index Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. | Research frontier | Checked automatically | 28.1% | In use |
| DeepTerminalBench Agents get one attempt at each of 50 deep engineering tasks in Docker, in the Terminal-Bench 2.0 format. A task passes only if every hidden test passes. Tasks were calibrated so Claude Opus 4.6 still fails some at pass@4. | Expert | Checked automatically | 34% | Not in cards |
| o11y-bench Agents query metrics, logs and traces in a live Grafana stack, investigate incidents and fix dashboards. Grading checks claims against facts read from the same environment. | Expert | Checked automatically | 93.8% | Not in cards |
| aws-bench Coding agents and model pairs diagnose misconfigurations, provision infrastructure and operate live cloud setups in disposable real AWS accounts. | Expert | Checked automatically | Not in cards | |
| ITBench-AA Agents read an offline snapshot of a failing Kubernetes cluster and name the entities that caused the incident, in structured JSON. | Expert | Graded by a model | Not in cards | |
| OpenThoughts-TBLite Agents solve 100 Terminal-Bench tasks picked to track Terminal-Bench 2 scores while running faster. Tasks are balanced across four difficulty bands using Claude Haiku 4.5 pass rates. | Expert | Checked automatically | Not in cards | |
| Terminal-Bench 2.0 Verified (Z.ai) Agents do the Terminal-Bench 2.0 tasks after Z.ai fixed environments for 89 tasks and instructions for 11. A further 2026-08-18 round fixed tests and instructions in 6 more tasks. | Expert | Checked automatically | Not in cards | |
| Terminal-Bench 2.1 Extended Agents complete long command-line tasks in containers, scored by verification tests. Mercor adds 99 new expert-written tasks to the 89 public ones and reports both scores. | Expert | Checked automatically | Not in cards | |
| Terminal-X A suite of three terminal datasets for coding agents. DeepTerminalBench tests single-shot depth, EvoCode-Bench tests multi-round iteration and RoadmapBench tests version upgrades on real repositories. | Expert | Checked automatically | Not in cards | |
| TUA-Bench Agents use a terminal for general computer tasks beyond coding, such as file, system and everyday productivity work. | Everyday | Checked automatically | Not in cards |
Long terminal sessions
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Long-Horizon-Terminal-Bench (LHTB) Terminal-Bench-style container tasks needing hundreds of steps. Subtasks graded deterministically for dense partial-credit reward. Solved at reward >=0.95. | Expert | Checked automatically | 60.2% | Not in cards |
| Long-Horizon-Terminal-Bench 2.0 Agents work for up to 3 hours per task on 77 container tasks, including 30 tasks that rebuild a scientific tool from its outputs. Hidden verifiers give partial-credit rewards. | Expert | Checked automatically | 69.2% | Not in cards |
| EdgeBench Agents iterate for up to 12 hours on 134 executable real-world tasks with multi-level feedback. Scores are tracked at 2 hour steps to measure learning from the environment. | Expert | Checked automatically | 51.3 | Not in cards |
| Terminal-Bench Challenges Agents build a large project end to end with no time limit, such as a faster Rust compiler or a WebGL renderer in WebAssembly. Each challenge has its own test bed and metric. | Research frontier | Checked automatically | Not in cards |