Benchmark Almanac

ExeWorldBench

Not in model cards

Coding agents build executable code-based 3D worlds that capture how a world works, testing world understanding and its realization in code.

77 upvotes on Hugging Face papers.

In the authors’ words

Coding agents are increasingly evolving beyond reproducing what a world looks like toward constructing executable worlds that capture how a world works. This shift calls for executable world modeling, which integrates two core capabilities: world understanding, reasoning about the underlying structural, mechanical, and causal principles of a world, and executable realization, translating that understanding into a code-based system whose behavior emerges from physical interactions. However, existing benchmarks provide only fragmented evaluations centered on physical reasoning, visual fidelity, or behavioral correctness, leaving the integration of these capabilities insufficiently assessed. To bridge this gap, we introduce ExeWorldBench, to our knowledge the first benchmark to comprehensively evaluate world understanding and executable realization through code-based 3D world construction. ExeWorldBench comprises three progressively challenging tasks: Mechanism Executability, which evaluates the realization of individual mechanisms from visual observations; Compositional Executability, which extends to multi-stage systems with spatial, mechanical, and causal dependencies; and Intent-Driven Executable World Synthesis, which requires constructing executable worlds from high-level goals and physical requirements. Extensive experiments on frontier models reveal substantial limitations in executable world modeling, offering insights into the development of more physically grounded and causally coherent world construction systems.

Abstract of ExeWorldBench: Benchmarking Code-Based Executable 3D World Modeling (arXiv:2610.15166, CC0)
Made by University of Science and Technology of China Released Oct 2026 Physical prediction All tasks public
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Code A program or patch run against tests.

No scores here yet

No lab, leaderboard or Epoch AI run has reported a score for ExeWorldBench yet.

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in World models

All 34 in World models
  • AgentWorldBench In use Language world models predict the next environment observation after an agent action, in 7 domains from MCP and search to Android and desktop OS. A judge model scores each prediction against the real observation on five dimensions. Qwen-AgentWorld-35B-A3B 56.4
  • IntPhys 2 In use Models watch simulated videos and judge whether each event is physically possible, testing object permanence, immutability, continuity and solidity. Scored by accuracy. Muse Spark Open 68.3%
  • QwenWorldBench Fading Models act as world models that simulate agent environments in 7 domains, such as terminal, web, and Android. A rubric judge scores five dimensions. Qwen3.7-Plus 62.1%
  • VPCT Fading Models view an image of a ball, ramps, and three buckets and predict which bucket the ball falls into. Scored by accuracy on 100 problems. Gemini 3 Pro 91%
  • ContPhy Fading Models infer physical properties such as mass and density of fluids and soft bodies from videos and predict their dynamics. Seed2.0 Pro 67.4%
  • MVPBench Dropped Models answer video questions on spatial, temporal and intuitive physics. Each question comes in a pair of near-identical videos with opposite answers, so text or single-frame shortcuts fail. gemini-robotics-er-1.5-preview 72.5%