Benchmark Almanac

Swarm Bench

Kimi K3 76.3% Only in system cards

Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution.

In-house Moonshot AI evaluation. The K2.5 report lists WildSearch, Batch Download, WideRead and Long-Form Writing domains. Tasks are not public.

Made by Moonshot AI Released Feb 2026 Multi-agent games Tasks kept private Last in a system card Jul 2026 Last model added Jul 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
2
Models scored in lab cards
2
System cards report it, from 1 lab

Kimi K3 leads with 76.3%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 76.3 Kimi K3
  2. 58.3 Kimi K2.5
  1. 1 Kimi K3 76.3%
  2. 2 Kimi K2.5 58.3%
#ModelScoreSettingDateSource
1 Kimi K3 76.3% Jul 2026 Model card
2 Kimi K2.5 58.3% Jan 2026 Model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Kimi K3 Kimi K2.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Kimi K3 Kimi K2.5

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

2 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
27 Jul 2026 Kimi K3 technical report 76.3 Swarm Bench, Kimi Agent harness (Table 3) Capabilities
30 Jan 2026 Kimi K2.5 technical report 58.3 In-house Swarm Bench, K2.5 Agent Swarm. Single-agent Kimi K2.5 41.6 (Table 6) Capabilities

More in Games and long tasks

All 68 in Games and long tasks
  • ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 100%
  • MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks. Seed2.0 Pro 49%
  • Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Scores come from in-game progress or outcomes per title. Nemotron 3 Nano 4B 22.9
  • GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. Gemini 3 Flash 41.9%
  • TextQuests Only in system cards Agents play 25 classic interactive fiction games. Measures game progress and harmful actions per 100 steps. Muse Spark 2.40
  • Anthropic large agent teams tasks Only in system cards Internal 24-hour tasks for teams of 1-100 agents: build a company knowledge base, or formalize a theorem in Lean.