Benchmark Almanac

LongAudioSpan

Models answer 3,240 questions about 360 English and Chinese recordings from 10 minutes to over 2 hours. Each question is scored as multiple choice and as open-ended.

Qwen reports accuracy, rubric and chain scores.

In the authors’ words

General audio comprehension now covers speech, sound, and music over durations from seconds to hours, driven by large audio-language models (LALMs) that are increasingly omni-modal. Yet the benchmarks that test them still rely on clips of seconds, where scores saturate and models converge; recent long-form efforts extend duration but evaluate long audio much as short clips are. We introduce LongAudioSpan, a benchmark that spans both duration and depth: it pairs audio from 10 minutes to over 2 hours with 3,240 questions across three cognitive levels, namely perception, understanding, and reasoning. Two paths supply the questions, differing in how question content is sourced and how ground truth is obtained. Native QA extracts questions from the audio's content, posing each as a multiple-choice item and an open-ended one graded by detailed rubrics. Anchor QA instead injects ground truth, planting acoustic anchors into the audio and building a perception-to-reasoning chain scored only to the first error. A fully automated pipeline constructs every item through structured captioning, QA generation, and adversarial critic feedback. Evaluating 12 LALMs on LongAudioSpan, we find the hard part comes before reasoning: distilling a few relevant facts from a long, redundant signal. This difficulty grows with audio length and falls hardest on perception, especially temporal grounding. LongAudioSpan is available at https://huggingface.co/datasets/holvan/LongAudioSpan.

Abstract of LongAudioSpan: Spanning the Duration and Depth of Audio Comprehension (arXiv:2608.26431, CC0)
Made by Huang et al. (arXiv 2608.26431, with Qwen authors) Released Aug 2026 Audio understanding and voice All tasks public Last in a launch post Sep 2026 Last model added Sep 2026
Level
Everyday Most adults, without training
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
1
Model scored in lab cards
1
Launch post lists it, from 1 lab
1
System card reports it, from 1 lab

Qwen3.8-Omni-Flash leads with 82.7%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 Qwen3.8-Omni-Flash 82.7%
#ModelScoreSettingDateSource
1 Qwen3.8-Omni-Flash 82.7% Sep 2026 Model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card
0% 20% 40% 60% 80% 100% Near the ceiling (85%) Qwen3.8-Omni-Flash 0% 20% 40% 60% 80% 100% Near the ceiling (85%) Qwen3.8-Omni-Flash

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
DateLaunch postScoreAs the post writes it
18 Sep 2026 Qwen3.8-Omni-Flash 82.7 LongAudioSpan Long Audio Reasoning Accuracy 82.7 Rubric 71.8 Chain 48.2

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
22 Sep 2026 Qwen3.8-Omni-Flash technical report 82.7 LongAudioSpan Accuracy, Rubric 71.8, Chain 48.2 (Table 4) Capabilities

More in Audio and video

All 168 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%
  • TVBench In use Models answer multiple-choice questions on 10 temporal video tasks, such as action counting and ordering, that static frames cannot solve. Scored by accuracy. Qwen 3.8 Max 81.9%