Benchmark Almanac

DECIDE

Tazo 67.6 tops its leaderboard Qwen2.5-Omni-7B 66.6 leads the models we track
Not in model cards

Models select decisions from short voice-call clips and conversation history across 4,867 questions, 1,081 clips, and 32 tasks. Human reference labels determine accuracy. The default board ranks by DECIDE Balance Score, which combines accuracy, cost, and latency on a zero-to-100 scale.

Tazo keeps the full benchmark private and publishes 20 labeled sample questions. Models can receive audio directly or use speech recognition before answering. Tazo reports optimized serving latency for its model and client-observed latency for others. Run costs include speech recognition when used.

Made by
Tazo Labs and Bolna
Released
November 2026
Kind
Audio understanding and voice
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Strongest model tested
Gemini 2.5 Pro Jun 2025 #96 on the capability index
Last model added
September 2026
Scores refreshed
November 12th, 2026
20
Entries on its own leaderboard, 8 models we track

Tazo tops the leaderboard with 67.6

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen2.5-Omni-7B, at 66.6.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

20 entries. Scroll sideways for the rest.

  1. 67.6 Tazo
  2. 66.6 Qwen2.5-Omni-7B
  3. 65.8 Qwen2.5 Omni 3B
  4. 61.5 STT + Jev
  5. 58.8 STT + Perplexity Decider Flash
  6. 57.5 STT + Liquid d1
  7. 56.5 STT + Tev1
  8. 55.2 Perceptron Mk1.5
  9. 52.3 Gemini 2.5 Flash-Lite
  10. 48.6 STT + Qwen3.5-9B
  11. 45.5 Voxtral Small 24B
  12. 40.3 MiMo-V2.6 Pro
  13. 36.6 Gemini 2.5 Flash
  14. 33.5 GPT-Realtime-2.1 Mini
  15. 30.8 GPT Audio
  16. 30.5 Qwen3.8-Omni-Flash
  17. 21.6 STT + GPT-4o
  18. 20.9 GPT-Realtime-2.1
  19. 18.0 Gemini 2.5 Pro
  20. 15.6 Meta Muse Spark
  1. 1 Tazo direct audio 67.6
  2. 2 Qwen2.5-Omni-7B direct audio 66.6
  3. 3 Qwen2.5 Omni 3B direct audio 65.8
  4. 4 STT + Jev speech-to-text input 61.5
  5. 5 STT + Perplexity Decider Flash speech-to-text input 58.8
  6. 6 STT + Liquid d1 speech-to-text input 57.5
  7. 7 STT + Tev1 speech-to-text input 56.5
  8. 8 Perceptron Mk1.5 direct audio 55.2
  9. 9 Gemini 2.5 Flash-Lite Sep 2025, direct audio 52.3
  10. 10 STT + Qwen3.5-9B speech-to-text input 48.6
  11. 11 Voxtral Small 24B direct audio 45.5
  12. 12 MiMo-V2.6 Pro direct audio 40.3
  13. 13 Gemini 2.5 Flash Sep 2025, direct audio 36.6
  14. 14 GPT-Realtime-2.1 Mini direct audio 33.5
  15. 15 GPT Audio direct audio 30.8
  16. 16 Qwen3.8-Omni-Flash direct audio 30.5
  17. 17 STT + GPT-4o speech-to-text input 21.6
  18. 18 GPT-Realtime-2.1 direct audio 20.9
  19. 19 Gemini 2.5 Pro Jun 2025, direct audio 18.0
  20. 20 Meta Muse Spark direct audio 15.6

Source: Official DECIDE leaderboard

#EntryScoreSettingDateSource
1 Tazo 67.6 direct audio Official leaderboard
2 Qwen2.5-Omni-7B 66.6 direct audio Mar 2025 Official leaderboard
3 Qwen2.5 Omni 3B 65.8 direct audio Official leaderboard
4 STT + Jev 61.5 speech-to-text input Official leaderboard
5 STT + Perplexity Decider Flash 58.8 speech-to-text input Official leaderboard
6 STT + Liquid d1 57.5 speech-to-text input Official leaderboard
7 STT + Tev1 56.5 speech-to-text input Official leaderboard
8 Perceptron Mk1.5 55.2 direct audio Official leaderboard
9 Gemini 2.5 Flash-Lite Sep 2025 52.3 direct audio Sep 2025 Official leaderboard
10 STT + Qwen3.5-9B 48.6 speech-to-text input Official leaderboard
11 Voxtral Small 24B 45.5 direct audio Official leaderboard
12 MiMo-V2.6 Pro 40.3 direct audio Sep 2026 Official leaderboard
13 Gemini 2.5 Flash Sep 2025 36.6 direct audio Sep 2025 Official leaderboard
14 GPT-Realtime-2.1 Mini 33.5 direct audio Jul 2026 Official leaderboard
15 GPT Audio 30.8 direct audio Official leaderboard
16 Qwen3.8-Omni-Flash 30.5 direct audio Sep 2026 Official leaderboard
17 STT + GPT-4o 21.6 speech-to-text input Official leaderboard
18 GPT-Realtime-2.1 20.9 direct audio Jul 2026 Official leaderboard
19 Gemini 2.5 Pro Jun 2025 18.0 direct audio Jun 2025 Official leaderboard
20 Meta Muse Spark 15.6 direct audio Official leaderboard

DECIDE efficiency–latency Pareto frontier

Models compare accuracy per dollar with latency using the board’s median-centered scales. The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

0 20 40 60 80 020406080100 Normalized latency (index), lower is better Normalized efficiency (index) Gemini 2.5 Pro (direct audio): Normalized efficiency 7.37 index, normalized latency 87.5 index Gemini 2.5 Flash-Lite (direct audio): Normalized efficiency 67.4 index, normalized latency 62.7 index Gemini 2.5 Flash (direct audio): Normalized efficiency 29.7 index, normalized latency 75.3 index GPT-Realtime-2.1 (direct audio): Normalized efficiency 6.8 index, normalized latency 79 index GPT-Realtime-2.1 Mini (direct audio): Normalized efficiency 24.9 index, normalized latency 76.5 index Qwen3.8-Omni-Flash (direct audio): Normalized efficiency 50.8 index, normalized latency 92 index MiMo-V2.6 Pro (direct audio): Normalized efficiency 49.2 index, normalized latency 80.5 index STT + Liquid d1 (speech-to-text input): Normalized efficiency 64.5 index, normalized latency 38.3 index. Not in our model list STT + Tev1 (speech-to-text input): Normalized efficiency 57.9 index, normalized latency 33.1 index. Not in our model list STT + Qwen3.5-9B (speech-to-text input): Normalized efficiency 48.8 index, normalized latency 51.6 index. Not in our model list Voxtral Small 24B (direct audio): Normalized efficiency 31.2 index, normalized latency 48.3 index. Not in our model list GPT Audio (direct audio): Normalized efficiency 7.43 index, normalized latency 36.8 index. Not in our model list STT + GPT-4o (speech-to-text input): Normalized efficiency 2.93 index, normalized latency 41 index. Not in our model list Meta Muse Spark (direct audio): Normalized efficiency 4.35 index, normalized latency 86 index. Not in our model list Qwen2.5-Omni-7B (direct audio): Normalized efficiency 59.7 index, normalized latency 22.8 index Tazo (direct audio): Normalized efficiency 58.2 index, normalized latency 21.3 index. Not in our model list Qwen2.5 Omni 3B (direct audio): Normalized efficiency 58.9 index, normalized latency 22.5 index. Not in our model list STT + Jev (speech-to-text input): Normalized efficiency 62.9 index, normalized latency 28.1 index. Not in our model list STT + Perplexity Decider Flash (speech-to-text input): Normalized efficiency 67 index, normalized latency 36.8 index. Not in our model list Perceptron Mk1.5 (direct audio): Normalized efficiency 69.6 index, normalized latency 62.6 index. Not in our model list Perceptron Mk1.5 (direct audio)STT + Perplexity Decider Flash (speech-to-text input)STT + Jev (speech-to-text input)Qwen2.5-Omni-7B (direct audio)Qwen2.5 Omni 3B (direct audio)Tazo (direct audio) 0 20 40 60 80 020406080100 Normalized latency (index), lower is better Normalized efficiency (index) Gemini 2.5 Pro (direct audio): Normalized efficiency 7.37 index, normalized latency 87.5 index Gemini 2.5 Flash-Lite (direct audio): Normalized efficiency 67.4 index, normalized latency 62.7 index Gemini 2.5 Flash (direct audio): Normalized efficiency 29.7 index, normalized latency 75.3 index GPT-Realtime-2.1 (direct audio): Normalized efficiency 6.8 index, normalized latency 79 index GPT-Realtime-2.1 Mini (direct audio): Normalized efficiency 24.9 index, normalized latency 76.5 index Qwen3.8-Omni-Flash (direct audio): Normalized efficiency 50.8 index, normalized latency 92 index MiMo-V2.6 Pro (direct audio): Normalized efficiency 49.2 index, normalized latency 80.5 index STT + Liquid d1 (speech-to-text input): Normalized efficiency 64.5 index, normalized latency 38.3 index. Not in our model list STT + Tev1 (speech-to-text input): Normalized efficiency 57.9 index, normalized latency 33.1 index. Not in our model list STT + Qwen3.5-9B (speech-to-text input): Normalized efficiency 48.8 index, normalized latency 51.6 index. Not in our model list Voxtral Small 24B (direct audio): Normalized efficiency 31.2 index, normalized latency 48.3 index. Not in our model list GPT Audio (direct audio): Normalized efficiency 7.43 index, normalized latency 36.8 index. Not in our model list STT + GPT-4o (speech-to-text input): Normalized efficiency 2.93 index, normalized latency 41 index. Not in our model list Meta Muse Spark (direct audio): Normalized efficiency 4.35 index, normalized latency 86 index. Not in our model list Qwen2.5-Omni-7B (direct audio): Normalized efficiency 59.7 index, normalized latency 22.8 index Tazo (direct audio): Normalized efficiency 58.2 index, normalized latency 21.3 index. Not in our model list Qwen2.5 Omni 3B (direct audio): Normalized efficiency 58.9 index, normalized latency 22.5 index. Not in our model list STT + Jev (speech-to-text input): Normalized efficiency 62.9 index, normalized latency 28.1 index. Not in our model list STT + Perplexity Decider Flash (speech-to-text input): Normalized efficiency 67 index, normalized latency 36.8 index. Not in our model list Perceptron Mk1.5 (direct audio): Normalized efficiency 69.6 index, normalized latency 62.6 index. Not in our model list Perceptron Mk1.5 (direct audio)STT + Perplexity Decider Flash (speech-to-text input)STT + Jev (speech-to-text input)Qwen2.5-Omni-7B (direct audio)Qwen2.5 Omni 3B (direct audio)Tazo (direct audio)
ModelSettingNormalized efficiencyNormalized latency
Perceptron Mk1.5 direct audio 69.61 index 62.62 index
Gemini 2.5 Flash-Lite Sep 2025 direct audio 67.41 index 62.75 index
STT + Perplexity Decider Flash speech-to-text input 67.02 index 36.78 index
STT + Liquid d1 speech-to-text input 64.48 index 38.35 index
STT + Jev speech-to-text input 62.92 index 28.08 index
Qwen2.5-Omni-7B direct audio 59.74 index 22.8 index
Qwen2.5 Omni 3B direct audio 58.9 index 22.45 index
Tazo direct audio 58.23 index 21.27 index
STT + Tev1 speech-to-text input 57.86 index 33.09 index
Qwen3.8-Omni-Flash direct audio 50.8 index 91.97 index
MiMo-V2.6 Pro direct audio 49.17 index 80.51 index
STT + Qwen3.5-9B speech-to-text input 48.81 index 51.59 index
Voxtral Small 24B direct audio 31.24 index 48.31 index
Gemini 2.5 Flash Sep 2025 direct audio 29.75 index 75.31 index
GPT-Realtime-2.1 Mini direct audio 24.92 index 76.46 index
GPT Audio direct audio 7.43 index 36.83 index
Gemini 2.5 Pro Jun 2025 direct audio 7.37 index 87.52 index
GPT-Realtime-2.1 direct audio 6.8 index 79.01 index
Meta Muse Spark direct audio 4.35 index 85.96 index
STT + GPT-4o speech-to-text input 2.93 index 41.03 index

The best score, 66.6, costs $0.0006 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Qwen Google DeepMind Cost frontier One model at each effort level
0.00 20.0 40.0 60.0 80.0 $0.0001 $0.001 $0.01 $0.10 Cost per task, log scale Qwen2.5-Omni-7B, direct audio Gemini 2.5 Flash-Lite, direct audio 0.00 20.0 40.0 60.0 80.0 $0.0001 $0.001 $0.01 $0.10 Cost per task, log scale Qwen2.5-Omni-7B, direct audio Gemini 2.5 Flash-Lite, direct audio

Source: Official DECIDE leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Qwen2.5-Omni-7B direct audio 66.6 $0.0006
Gemini 2.5 Flash-Lite Sep 2025 direct audio 52.3 $0.0003
MiMo-V2.6 Pro direct audio 40.3 $0.001
Gemini 2.5 Flash Sep 2025 direct audio 36.6 $0.002
GPT-Realtime-2.1 Mini direct audio 33.5 $0.003
Qwen3.8-Omni-Flash direct audio 30.5 $0.0009
GPT-Realtime-2.1 direct audio 20.9 $0.01
Gemini 2.5 Pro Jun 2025 direct audio 18.0 $0.01

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen
Own leaderboard 1 leader
Qwen2.5-Omni-7B 66.6

Source: Official DECIDE leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
20.0 40.0 60.0 2025 2026 20.0 40.0 60.0 2025 2026

Source: Official DECIDE leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Audio and video

All 232 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%