Benchmark Almanac

TriviaQA

Llama 3.1-405B 91.8% leads since Jul 2024 Fading

Trivia questions with evidence documents. Scored by exact match.

Saturated base-model metric.

In the authors’ words

We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions. We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers. We also present two baseline algorithms: a feature-based classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension. Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that TriviaQA is a challenging testbed that is worth significant future study. Data and code available at – http://nlp.cs.washington.edu/triviaqa/

Abstract of TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension (arXiv:1705.03551, CC0)
Made by University of Washington Released May 2017 Trivia and open questions All tasks public Last in a launch post Apr 2026 , dropped after it saturated Last model added Apr 2026 Scores refreshed 31 Oct 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
mandarjoshi90/triviaqa 341 stars Paper Dataset
39
Models scored in lab cards
19
Launch posts list it, from 9 labs
31
System cards report it, from 9 labs

Llama 3.1-405B leads with 91.8% among models released by Apr 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Apr 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

39 models. Scroll sideways for the rest.

  1. 91.8 Llama 3.1-405B
  2. 91.0 Sarvam-M
  3. 89.8 Llama 3.1-70B
  4. 89.7 Llama 3-70B
  5. 87.6 Llama 2-70B
  6. 87.5 Claude 2
  7. 86.7 DeepSeek-V2 (MoE-236B, May 2024)
  8. 86.1 PaLM 2-L
  9. 85.6 DeepSeek-V4-Pro
  10. 85.5 Gemma 3 27B
  11. 84.5 Llama 2-34B
  12. 83.7 Gemma 2 27B
  13. 82.9 DeepSeek-V3
  14. 82.7 Mistral Large
  15. 81.5 DeepSeek LLM 67B
  16. 80.9 GLM-5
  17. 80.3 Mistral Small 3
  18. 80.0 GLM-4.5
  19. 79.6 Llama 2-13B
  20. 78.6 Ring-1T
  21. 78.5 Llama 3-8B
  22. 78.2 Gemma 3 12B
  23. 77.6 Llama 3.1-8B
  24. 76.6 Gemma 2 9B
  25. 75.6 phi-3-medium 14B
  26. 74.8 InternVL3.5-241B-A28B
  27. 72.1 Llama 2-7B
  28. 71.5 Mixtral 8x7B
  29. 70.2 Gemma 3n E4B
  30. 69.9 Mistral 7B v0.1
  31. 66.3 Moonlight-16B-A3B
  32. 65.8 Gemma 3 4B
  33. 64.0 phi-3-mini 3.8B
  34. 63.4 Gemma 7B
  35. 58.1 phi-3-small 7.4B
  36. 53.2 Gemma 2B
  37. 47.7 AFM-4.5B
  38. 39.8 Gemma 3 1B
  39. 33.6 BitNet b1.58 2B4T
  1. 1 Llama 3.1-405B 91.8%
  2. 2 Sarvam-M 91%
  3. 3 Llama 3.1-70B 89.8%
  4. 4 Llama 3-70B 89.7%
  5. 5 Llama 2-70B 87.6%
  6. 6 Claude 2 87.5%
  7. 7 DeepSeek-V2 (MoE-236B, May 2024) 86.7%
  8. 8 PaLM 2-L 86.1%
  9. 9 DeepSeek-V4-Pro 85.6%
  10. 10 Gemma 3 27B 85.5%
  11. 11 Llama 2-34B 84.5%
  12. 12 Gemma 2 27B 83.7%
  13. 13 DeepSeek-V3 82.9%
  14. 14 Mistral Large 82.7%
  15. 15 DeepSeek LLM 67B 81.5%
  16. 16 GLM-5 80.9%
  17. 17 Mistral Small 3 80.3%
  18. 18 GLM-4.5 80%
  19. 19 Llama 2-13B 79.6%
  20. 20 Ring-1T 78.6%
  21. 21 Llama 3-8B 78.5%
  22. 22 Gemma 3 12B 78.2%
  23. 23 Llama 3.1-8B 77.6%
  24. 24 Gemma 2 9B 76.6%
  25. 25 phi-3-medium 14B 75.6%
  26. 26 InternVL3.5-241B-A28B 74.8%
  27. 27 Llama 2-7B 72.1%
  28. 28 Mixtral 8x7B 71.5%
  29. 29 Gemma 3n E4B 70.2%
  30. 30 Mistral 7B v0.1 69.9%
  31. 31 Moonlight-16B-A3B 66.3%
  32. 32 Gemma 3 4B 65.8%
  33. 33 phi-3-mini 3.8B 64%
  34. 34 Gemma 7B 63.4%
  35. 35 phi-3-small 7.4B 58.1%
  36. 36 Gemma 2B 53.2%
  37. 37 AFM-4.5B 47.7%
  38. 38 Gemma 3 1B 39.8%
  39. 39 BitNet b1.58 2B4T 33.6%
#ModelScoreSettingDateSource
1 Llama 3.1-405B 91.8% Jul 2024 Model card
2 Sarvam-M 91% May 2025 Model card
3 Llama 3.1-70B 89.8% Jul 2024 Model card
4 Llama 3-70B 89.7% Apr 2024 Model card
5 Llama 2-70B 87.6% Jul 2023 Model card
6 Claude 2 87.5% Jul 2023 Model card
7 DeepSeek-V2 (MoE-236B, May 2024) 86.7% May 2024 Model card
8 PaLM 2-L 86.1% May 2023 Model card
9 DeepSeek-V4-Pro 85.6% Apr 2026 Model card
10 Gemma 3 27B 85.5% Mar 2025 Model card
11 Llama 2-34B 84.5% Jul 2023 Model card
12 Gemma 2 27B 83.7% Jun 2024 Model card
13 DeepSeek-V3 82.9% Dec 2024 Model card
14 Mistral Large 82.7% Feb 2024 Model card
15 DeepSeek LLM 67B 81.5% Nov 2023 Model card
16 GLM-5 80.9% Feb 2026 Model card
17 Mistral Small 3 80.3% Jan 2025 Model card
18 GLM-4.5 80% Aug 2025 Model card
19 Llama 2-13B 79.6% Jul 2023 Model card
20 Ring-1T 78.6% Oct 2025 Model card
21 Llama 3-8B 78.5% Apr 2024 Model card
22 Gemma 3 12B 78.2% Mar 2025 Model card
23 Llama 3.1-8B 77.6% Jul 2024 Model card
24 Gemma 2 9B 76.6% Jul 2024 Model card
25 phi-3-medium 14B 75.6% Apr 2024 Model card
26 InternVL3.5-241B-A28B 74.8% Aug 2025 Model card
27 Llama 2-7B 72.1% Jul 2023 Model card
28 Mixtral 8x7B 71.5% Dec 2023 Model card
29 Gemma 3n E4B 70.2% Jun 2025 Model card
30 Mistral 7B v0.1 69.9% Sep 2023 Model card
31 Moonlight-16B-A3B 66.3% Feb 2025 Model card
32 Gemma 3 4B 65.8% Mar 2025 Model card
33 phi-3-mini 3.8B 64% Aug 2024 Model card
34 Gemma 7B 63.4% Feb 2024 Model card
35 phi-3-small 7.4B 58.1% Aug 2024 Model card
36 Gemma 2B 53.2% Mar 2024 Model card
37 AFM-4.5B 47.7% Jul 2025 Model card
38 Gemma 3 1B 39.8% Mar 2025 Model card
39 BitNet b1.58 2B4T 33.6% Apr 2025 Model card

Epoch AI has scores for 38 models. We plot every source in the charts below.

5 models have held number one since 2023

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for Epoch AI’s runs and what labs claim in their cards.

Google DeepMind Meta Anthropic Other labs
Epoch AI runs Passed 85% in Feb 2023, by LLaMA-65B
Llama 2-70B 87.6%
Lab claims Passed 85% in May 2023, by PaLM 2-L
Llama 3.1-405B 91.8%

The best score passed 85% in Feb 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Run or collected by Epoch AI Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2022 2023 2024 2025 2026 Near the ceiling (85%) Llama 3.1-405B Llama 2-70B PaLM (540B) GLaM Gopher (280B) 0% 20% 40% 60% 80% 100% 2022 2023 2024 2025 2026 Near the ceiling (85%) Llama 3.1-405B PaLM (540B)

Labs headline it less often now, last in Apr 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Anthropic
Mistral AI
DeepSeek
Meta
Microsoft
Moonshot AI
Sarvam AI
Arcee AI
DateLaunch postScoreAs the post writes it
24 Apr 2026 DeepSeek-V4-Pro 85.6 TriviaQA (EM)
29 Jul 2025 AFM-4.5B 47.7 TriviaQA
26 Jun 2025 Gemma 3n E4B 70.2 TriviaQA 5-shot (PT)
23 May 2025 Sarvam-M 91 TriviaQA (0.91)
15 Apr 2025 BitNet b1.58 2B4T 33.6 TriviaQA
12 Mar 2025 Gemma 3 27B 85.5 TriviaQA 5-shot (PT)
22 Feb 2025 Moonlight-16B-A3B 66.3 TriviaQA
30 Jan 2025 Mistral Small 3 80.3 TriviaQA (5-shot) [base; listed twice, 80.32 and 76.6]
26 Dec 2024 DeepSeek-V3 82.9 TriviaQA (EM)
27 Jun 2024 Gemma 2 27B 83.7 TriviaQA (5-shot)
22 Apr 2024 phi-3-medium 14B 75.6 TriviaQA (5-Shot)
18 Apr 2024 Llama 3-70B 89.7 TriviaQA-Wiki (5-shot) [pretrained]
26 Feb 2024 Mistral Large 82.7 TriQA (5-shot)
21 Feb 2024 Gemma 7B 63.4 TriviaQA (5-shot)
11 Dec 2023 Mixtral 8x7B 71.5 TriQA [detailed table]
29 Nov 2023 DeepSeek LLM 67B 81.5 TriviaQA
27 Sep 2023 Mistral 7B v0.1 69.9 TriviaQA
11 Jul 2023 Claude 2 87.5 TriviaQA (5-shot)
10 May 2023 PaLM 2-L 86.1 TriviaQA (EM), 1-shot

31 system cards report it, from 9 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
17 Feb 2026 GLM-5 technical report 80.9 GLM-5-Base, TriviaQA (EM) (Table 11) Capabilities
21 Oct 2025 Ring-1T technical report 78.6 TriviaQA (Table 1) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 74.8 TriviaQA, OpenCompass (Table 14) Capabilities
8 Aug 2025 GLM-4.5 technical report 80 GLM-4.5-Base, TriviaQA (EM) (Table 2) Capabilities
17 Jul 2025 Voxtral Small technical report Trivia QA 5-shot, text-only, chart without printed numbers (Figure 6). Speech-synthesized Trivia QA* 79.4 (Table 8) Capabilities
16 Apr 2025 BitNet b1.58 2B4T technical report 33.6 TriviaQA (5-shot; EM) (Table 1, Table 3) Capabilities
25 Mar 2025 Gemma 3 4B technical report 65.8 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
25 Mar 2025 Gemma 3 27B technical report 85.5 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
25 Mar 2025 Gemma 3 1B technical report 39.8 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
25 Mar 2025 Gemma 3 12B technical report 78.2 TriviaQA (TQA), PT, 5-shot (Table 9) Capabilities
30 Aug 2024 phi-3-small 7.4B technical report 58.1 TriviaQA (5-Shot) (Section 3 table) Capabilities
30 Aug 2024 phi-3-mini 3.8B technical report 64 TriviaQA (5-Shot) (Section 3 table) Capabilities
30 Aug 2024 phi-3-medium 14B technical report 73.9 TriviaQA (5-Shot) (Section 3 table) Capabilities
31 Jul 2024 Gemma 2 9B technical report 76.6 TriviaQA, PT, 5-shot (Table 13) Capabilities
31 Jul 2024 Gemma 2 27B technical report 83.7 TriviaQA, PT, 5-shot (Table 13) Capabilities
23 Jul 2024 Llama 3.1-8B model card 77.6 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
23 Jul 2024 Llama 3.1-70B model card 89.8 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
23 Jul 2024 Llama 3.1-405B model card 91.8 TriviaQA-Wiki, 5-shot, em, pretrained (Base pretrained models table) Capabilities
7 May 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 86.7 TriviaQA 5-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
18 Apr 2024 Llama 3-8B model card 78.5 TriviaQA-Wiki (5-shot), pretrained (Base pretrained models table) Capabilities
18 Apr 2024 Llama 3-70B model card 89.7 TriviaQA-Wiki (5-shot), pretrained (Base pretrained models table) Capabilities
13 Mar 2024 Gemma 7B technical report 63.4 TriviaQA, PT, 5-shot (Table 6) Capabilities
13 Mar 2024 Gemma 2B technical report 53.2 TriviaQA, PT, 5-shot (Table 6) Capabilities
8 Jan 2024 Mixtral 8x7B technical report 71.5 TriQA, 5-shot, no Wikipedia contexts (Table 2) Capabilities
5 Jan 2024 DeepSeek LLM 67B technical report 81.5 TriviaQA, DeepSeek 67B Chat (Table 6) Capabilities
10 Oct 2023 Mistral 7B v0.1 technical report 69.9 TriviaQA, 5-shot, no Wikipedia contexts (Table 2) Capabilities
19 Jul 2023 Llama 2-7B technical report 72.1 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 68.9 Capabilities
19 Jul 2023 Llama 2-70B technical report 87.6 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 85.0 Capabilities
19 Jul 2023 Llama 2-34B technical report 84.5 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 83.3 Capabilities
19 Jul 2023 Llama 2-13B technical report 79.6 TriviaQA (Wiki), exact match, 5-shot, pretrained (Table 22). 1-shot = 77.2 Capabilities
11 Jul 2023 Claude 2 model card 87.5 TriviaQA (5-shot) (4.3) Capabilities

More in Knowledge

All 109 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • BIG-Bench Extra Hard In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is a harmonic mean of task accuracies. Gemma 4 31B IT 74.4%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%