AgentWorldBench
Qwen-AgentWorld-35B-A3B 56.4 In use Language world models predict the next environment observation after an agent action, in 7 domains from MCP and search to Android and desktop OS. A judge model scores each prediction against the real observation on five dimensions.
2,170 samples built from real trajectories on Toolathlon, Terminal-Bench 1.0 and 2.0, and OSWorld-Verified. The judge is gpt-5.2. It looks like a public form of the in-house QwenWorldBench (same 7 domains and 5 rubric dimensions), but the scores differ, so it is kept separate.
- Level
- University Undergraduates and entry-level professionals
- Scored
- Graded by a model A language model scores each response, usually against a rubric.
- Answers
- Open-ended Free-form text, documents, designs or media judged as a whole.
Qwen-AgentWorld-35B-A3B leads with 56.4
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Qwen-AgentWorld-35B-A3B | 56.4 | Jun 2026 | Model card |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 22 Jun 2026 | Qwen-AgentWorld-35B-A3B | 56.4 | AgentWorldBench Overall (five-dimension rubric mean, 0 to 100) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 23 Jun 2026 | Qwen-AgentWorld-35B-A3B technical report | 56.4 | AgentWorldBench Avg., five-dimension rubric mean 0 to 100 (Table 5). MCP 64.79, Search 36.69, Terminal 53.96, SWE 65.63, Android 58.17, Web 49.55, OS 65.92. Rule-based verification average 57.28 % (Table 10) | Capabilities |
More in World models
- IntPhys 2 In use Models watch simulated videos and judge whether each event is physically possible, testing object permanence, immutability, continuity and solidity. Scored by accuracy.
Muse Spark Open 68.3% - QwenWorldBench Fading Models act as world models that simulate agent environments in 7 domains, such as terminal, web, and Android. A rubric judge scores five dimensions. Qwen3.7-Plus 62.1%
- VPCT Fading Models view an image of a ball, ramps, and three buckets and predict which bucket the ball falls into. Scored by accuracy on 100 problems.
Gemini 3 Pro 91% - ContPhy Fading Models infer physical properties such as mass and density of fluids and soft bodies from videos and predict their dynamics.
Seed2.0 Pro 67.4% - MVPBench Dropped Models answer video questions on spatial, temporal and intuitive physics. Each question comes in a pair of near-identical videos with opposite answers, so text or single-frame shortcuts fail. gemini-robotics-er-1.5-preview 72.5%
- RBench Not in model cards 650 image+text prompts for robot video generation (tasks and embodiments). MLLM judge scores task completion, physical plausibility, stability.
Wan 2.6 60.7%