MVPBench
gemini-robotics-er-1.5-preview 72.5% Dropped Models answer video questions on spatial, temporal and intuitive physics. Each question comes in a pair of near-identical videos with opposite answers, so text or single-frame shortcuts fail.
Released with V-JEPA 2. The leaderboard uses the mini split of about 5,700 items. Videos come from nine public datasets.
- Level
- Everyday Most adults, without training
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Multiple choice
gemini-robotics-er-1.5-preview leads with 72.5%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 gemini-robotics-er-1.5-preview 72.5%
- 2
V-JEPA 2 44.5%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | gemini-robotics-er-1.5-preview | 72.5% | Sep 2025 | Model card | |
| 2 | V-JEPA 2 | 44.5% | Jun 2025 | Model card |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
No launch post has listed it since Jun 2025
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 11 Jun 2025 | V-JEPA 2 | 44.5 | MVP (Video QA) |
One system card reports it
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 25 Sep 2025 | gemini-robotics-er-1.5-preview technical report | 72.5 | Min Video Pairs (Table 19, thinking on). Thinking off 67.1 | Capabilities |
More in World models
- AgentWorldBench In use Language world models predict the next environment observation after an agent action, in 7 domains from MCP and search to Android and desktop OS. A judge model scores each prediction against the real observation on five dimensions.
Qwen-AgentWorld-35B-A3B 56.4 - IntPhys 2 In use Models watch simulated videos and judge whether each event is physically possible, testing object permanence, immutability, continuity and solidity. Scored by accuracy. Muse Spark Open 68.3%
- QwenWorldBench Fading Models act as world models that simulate agent environments in 7 domains, such as terminal, web, and Android. A rubric judge scores five dimensions. Qwen3.7-Plus 62.1%
- VPCT Fading Models view an image of a ball, ramps, and three buckets and predict which bucket the ball falls into. Scored by accuracy on 100 problems. Gemini 3 Pro 91%
- ContPhy Fading Models infer physical properties such as mass and density of fluids and soft bodies from videos and predict their dynamics.
Seed2.0 Pro 67.4% - RBench Not in model cards 650 image+text prompts for robot video generation (tasks and embodiments). MLLM judge scores task completion, physical plausibility, stability.
Wan 2.6 60.7%