MSRVTT-QA
Models answer short open-ended questions about web video clips from the MSR-VTT captioning dataset. The question and answer pairs were generated automatically from the clip descriptions. Scored by answer accuracy.
The repository also holds the sibling MSVD-QA set and the authors' model, and its 177 stars cover all three. Video language model papers report it often, but many now grade answers with a GPT judge, so scores vary by protocol.
- Made by
- Zhejiang University and collaborators (Xu et al.)
- Released
- October 2017
- Kind
- Video understanding
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
No scores here yet
No lab, leaderboard or Epoch AI run has reported a score for MSRVTT-QA yet.
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Audio and video
- Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy.
Qwen 3.8 Max 88.7% - LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization.
Gemini 4 Argon 91.7% - MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
- MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
- FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better.
Phi-4-multimodal 4.00% - VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%