Benchmark Almanac

FutureOmni

MiMo-V2-Omni 66.7% Fading

Models watch and hear 919 videos and answer 1,034 multiple-choice questions that ask what will happen next. Questions need audio and visual cues together.

Built with an LLM-assisted, human-in-the-loop pipeline.

In the authors’ words

Although Multimodal Large Language Models (MLLMs) demonstrate strong omni-modal perception, their ability to forecast future events from audio-visual cues remains largely unexplored, as existing benchmarks focus mainly on retrospective understanding. To bridge this gap, we introduce FutureOmni, the first benchmark designed to evaluate omni-modal future forecasting from audio-visual environments. The evaluated models are required to perform cross-modal causal and temporal reasoning, as well as effectively leverage internal knowledge to predict future events. FutureOmni is constructed via a scalable LLM-assisted, human-in-the-loop pipeline and contains 919 videos and 1,034 multiple-choice QA pairs across 8 primary domains. Evaluations on 13 omni-modal and 7 video-only models show that current systems struggle with audio-visual future prediction, particularly in speech-heavy scenarios, with the best accuracy of 64.8% achieved by Gemini 3 Flash. To mitigate this limitation, we curate a 7K-sample instruction-tuning dataset and propose an Omni-Modal Future Forecasting (OFF) training strategy. Evaluations on FutureOmni and popular audio-visual and video-only benchmarks demonstrate that OFF enhances future forecasting and generalization. We publicly release all code (https://github.com/OpenMOSS/FutureOmni) and datasets (https://huggingface.co/datasets/OpenMOSS-Team/FutureOmni).

Abstract of FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs (arXiv:2601.13836, CC0)
Made by OpenMOSS (Fudan University) and collaborators Released Jan 2026 Video understanding All tasks public Last in a launch post Mar 2026 , dropped before it saturated Last model added Mar 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
1
Model scored in lab cards
1
Launch post lists it, from 1 lab

MiMo-V2-Omni leads with 66.7%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 MiMo-V2-Omni 66.7%
#ModelScoreSettingDateSource
1 MiMo-V2-Omni 66.7% Mar 2026 Model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) MiMo-V2-Omni 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) MiMo-V2-Omni

Labs headline it less often now, last in Mar 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Xiaomi Corp
DateLaunch postScoreAs the post writes it
18 Mar 2026 MiMo-V2-Omni 66.7 FutureOmni

More in Audio and video

All 168 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%
  • TVBench In use Models answer multiple-choice questions on 10 temporal video tasks, such as action counting and ordering, that static frames cannot solve. Scored by accuracy. Qwen 3.8 Max 81.9%