DeepTerminalBench
Agents get one attempt at each of 50 deep engineering tasks in Docker, in the Terminal-Bench 2.0 format. A task passes only if every hidden test passes. Tasks were calibrated so Claude Opus 4.6 still fails some at pass@4.
Runs on Harbor with the terminus-2 agent and a 90-minute limit per task. The first release covered 9 frontier models. Tasks span more than 20 engineering domains.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
Claude Opus 4.7 leads with 34% among models released by Apr 2026
Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.
The newest model with a score here came out in Apr 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 15 more
- 1 Claude Opus 4.7 34%
- 2
Qwen 3.6 Plus 32% - 3 Kimi K2.6 32%
- 4 GPT-5.5 high 32%
- 5
DeepSeek-V4-Pro 32% - 6 GLM-5.1 30%
- 7 Qwen3.5 397B-A17B 24%
- 8
Gemini 3.1 Pro 16% - 9
MiniMax-M2.7 10%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.7 | 34% | terminus-2 | Apr 2026 | Official leaderboard |
| 2 | Qwen 3.6 Plus | 32% | terminus-2 | Mar 2026 | Official leaderboard |
| 3 | Kimi K2.6 | 32% | terminus-2 | Apr 2026 | Official leaderboard |
| 4 | GPT-5.5 | 32% | high, terminus-2 | Apr 2026 | Official leaderboard |
| 5 | DeepSeek-V4-Pro | 32% | terminus-2 | Apr 2026 | Official leaderboard |
| 6 | GLM-5.1 | 30% | terminus-2 | Apr 2026 | Official leaderboard |
| 7 | Qwen3.5 397B-A17B | 24% | terminus-2 | Feb 2026 | Official leaderboard |
| 8 | Gemini 3.1 Pro | 16% | terminus-2 | Feb 2026 | Official leaderboard |
| 9 | MiniMax-M2.7 | 10% | terminus-2 | Mar 2026 | Official leaderboard |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
No lab model card reports it yet
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.
More in Terminal agents
- Terminal-Bench 4.0 In use Agents complete 66 hard tasks in containerized terminals across software, ML, science, security, and operations. Test scripts verify each result, scored as resolution rate. Claude Opus 5.5 64.8%
- SkillsBench In use 86-87 containerized expert tasks in 11 domains, run with and without Agent Skills. Verifier tests give pass rate and skill lift. GPT-5.5 67.3%
- Terminal-Bench Hard In use Agents complete 44 hard tasks from Terminal-Bench Core in the Terminus 2 harness. Artificial Analysis scores pass@1 averaged over 3 repeats. Gemma 4 31B IT 36%
- Harbor-Index In use Agents solve about 80 hard tasks drawn from 29 benchmarks run through Harbor adapters, such as HLE, GSO and SWE-bench Pro. Each task keeps its source verifier. GPT-5.5 28.1%
- CompileBench Not in model cards Agents build real open-source projects through dependency problems, legacy toolchains and complex build systems, for example compiling 22-year-old code. Claude Opus 4.1 100%
- Long-Horizon-Terminal-Bench 2.0 Not in model cards Agents work for up to 3 hours per task on 77 container tasks, including 30 tasks that rebuild a scientific tool from its outputs. Hidden verifiers give partial-credit rewards. GPT-6 Astra 69.2%