TruthfulQA
MAI-Thinking-1 88% leads since Jun 2026 Fading Models answer 817 questions across 38 categories that are written to elicit common misconceptions. Scoring checks for truthful answers, often in multiple-choice form.
In January 2025 the authors added a two-option multiple-choice format and now recommend it. They say MC1 and MC2 results stay valid, so it remains one record.
In the authors’ words
We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or misconception. To perform well, models must avoid generating false answers learned from imitating human texts. We tested GPT-3, GPT-Neo/J, GPT-2 and a T5-based model. The best model was truthful on 58% of questions, while human performance was 94%. Models generated many false answers that mimic popular misconceptions and have the potential to deceive humans. The largest models were generally the least truthful. This contrasts with other NLP tasks, where performance improves with model size. However, this result is expected if false answers are learned from the training distribution. We suggest that scaling up models alone is less promising for improving truthfulness than fine-tuning using training objectives other than imitation of text from the web.
- Level
- Everyday Most adults, without training
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Multiple choice
MAI-Thinking-1 leads with 88% among models released by Jun 2026
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
The newest model with a score here came out in Jun 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 88.0 MAI-Thinking-1
- 77.5 Phi-3.5-MoE
- 75.7 phi-3-medium 14B
- 73.2
OLMo 2 32B Instruct - 70.2 phi-3-small 7.4B
- 69.0 Claude 2
- 66.9 Granite 3.2 8B Instruct
- 66.9 Granite 3.3 8B Instruct
- 66.4 Phi-4 Mini
- 65.3
Falcon-H1-34B - 65.0 phi-3-mini 3.8B
- 64.0 Phi-3.5-mini
- 61.2
Apertus 70B - 60.4
Qwen2.5-72B - 60.0 GPT-4
- 54.8 Qwen2-72B
- 51.6
Gemma 2 27B - 50.5
Mistral Large - 50.3 Gemma 2 9B
- 50.2
Llama 2-70B - 48.7 OLMo 2 1B Instruct
- 45.3 BitNet b1.58 2B4T
- 44.8 Gemma 2B
- 43.5 Llama 2-34B
- 41.9 Llama 2-13B
- 33.3 Llama 2-7B
- 31.8 Gemma 7B
- 31.5 Falcon-Arabic-7B
- 1 MAI-Thinking-1 88%
- 2 Phi-3.5-MoE 77.5%
- 3 phi-3-medium 14B 75.7%
- 4 OLMo 2 32B Instruct 73.2%
- 5 phi-3-small 7.4B 70.2%
- 6 Claude 2 69%
- 7 Granite 3.2 8B Instruct 66.9%
- 8 Granite 3.3 8B Instruct 66.9%
- 9 Phi-4 Mini 66.4%
- 10 Falcon-H1-34B 65.3%
- 11 phi-3-mini 3.8B 65%
- 12 Phi-3.5-mini 64%
- 13 Apertus 70B 61.2%
- 14 Qwen2.5-72B 60.4%
- 15 GPT-4 Mar 2023 60%
- 16 Qwen2-72B 54.8%
- 17 Gemma 2 27B 51.6%
- 18 Mistral Large 50.5%
- 19 Gemma 2 9B 50.3%
- 20 Llama 2-70B 50.2%
- 21 OLMo 2 1B Instruct 48.7%
- 22 BitNet b1.58 2B4T 45.3%
- 23 Gemma 2B 44.8%
- 24 Llama 2-34B 43.5%
- 25 Llama 2-13B 41.9%
- 26 Llama 2-7B 33.3%
- 27 Gemma 7B 31.8%
- 28 Falcon-Arabic-7B 31.5%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | MAI-Thinking-1 | 88% | Jun 2026 | Model card | |
| 2 | Phi-3.5-MoE | 77.5% | Aug 2024 | Model card | |
| 3 | phi-3-medium 14B | 75.7% | Apr 2024 | Model card | |
| 4 | OLMo 2 32B Instruct | 73.2% | Mar 2025 | Model card | |
| 5 | phi-3-small 7.4B | 70.2% | Aug 2024 | Model card | |
| 6 | Claude 2 | 69% | Jul 2023 | Model card | |
| 7 | Granite 3.2 8B Instruct | 66.9% | Feb 2025 | Model card | |
| 8 | Granite 3.3 8B Instruct | 66.9% | Apr 2025 | Model card | |
| 9 | Phi-4 Mini | 66.4% | Feb 2025 | Model card | |
| 10 | Falcon-H1-34B | 65.3% | May 2025 | Model card | |
| 11 | phi-3-mini 3.8B | 65% | Aug 2024 | Model card | |
| 12 | Phi-3.5-mini | 64% | Aug 2024 | Model card | |
| 13 | Apertus 70B | 61.2% | Sep 2025 | Model card | |
| 14 | Qwen2.5-72B | 60.4% | Sep 2024 | Model card | |
| 15 | GPT-4 Mar 2023 | 60% | Mar 2023 | Model card | |
| 16 | Qwen2-72B | 54.8% | Jun 2024 | Model card | |
| 17 | Gemma 2 27B | 51.6% | Jul 2024 | Model card | |
| 18 | Mistral Large | 50.5% | Feb 2024 | Model card | |
| 19 | Gemma 2 9B | 50.3% | Jul 2024 | Model card | |
| 20 | Llama 2-70B | 50.2% | Jul 2023 | Model card | |
| 21 | OLMo 2 1B Instruct | 48.7% | Apr 2025 | Model card | |
| 22 | BitNet b1.58 2B4T | 45.3% | Apr 2025 | Model card | |
| 23 | Gemma 2B | 44.8% | Mar 2024 | Model card | |
| 24 | Llama 2-34B | 43.5% | Jul 2023 | Model card | |
| 25 | Llama 2-13B | 41.9% | Jul 2023 | Model card | |
| 26 | Llama 2-7B | 33.3% | Jul 2023 | Model card | |
| 27 | Gemma 7B | 31.8% | Mar 2024 | Model card | |
| 28 | Falcon-Arabic-7B | 31.5% | Nov 2025 | Model card |
5 models have held number one since 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
The best score passed 85% in Jun 2026
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Jun 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 2 Jun 2026 | MAI-Thinking-1 | 88 | Truthful QA |
| 20 May 2025 | Falcon-H1-34B | 65.3 | TruthfulQA |
| 29 Apr 2025 | OLMo 2 1B Instruct | 48.7 | TruthQA |
| 16 Apr 2025 | Granite 3.3 8B Instruct | 66.9 | TruthfulQA |
| 15 Apr 2025 | BitNet b1.58 2B4T | 45.3 | TruthfulQA |
| 13 Mar 2025 | OLMo 2 32B Instruct | 73.2 | TruthQA |
| 26 Feb 2025 | Phi-4 Mini | 66.4 | TruthfulQA (MC2) (10-shot) |
| 26 Feb 2025 | Granite 3.2 8B Instruct | 66.9 | TruthfulQA |
| 19 Sep 2024 | Qwen2.5-72B | 60.4 | TruthfulQA |
| 20 Aug 2024 | Phi-3.5-MoE | 77.5 | TruthfulQA (MC2) (10-shot) |
| 7 Jun 2024 | Qwen2-72B | 54.8 | TruthfulQA |
| 22 Apr 2024 | phi-3-medium 14B | 75.7 | TruthfulQA (10-Shot) |
| 26 Feb 2024 | Mistral Large | 50.5 | TruthfulQA |
26 system cards report it, from 8 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 2 Jun 2026 | MAI-Thinking-1 technical report | 88 | Truthful QA, multiple-choice accuracy, Honesty (Table 12, rounded) | Hallucination |
| 8 Nov 2025 | Falcon-Arabic-7B technical report | 31.5 | TruthfulQA 0-shot (Table 8) | Hallucination |
| 8 Oct 2025 | OLMo 2 1B Instruct technical report | 48.7 | TQA, TruthfulQA (Table 7, Table 23) | Hallucination |
| 17 Sep 2025 | Apertus 70B technical report | 61.2 | TruthQA (TruthfulQA mc2) (Table 17). TruthQA Multilingual 53.7 | Capabilities |
| 30 Jul 2025 | Falcon-H1-34B technical report | 65.3 | TruthfulQA, logprobs 0-shot (Table 24) | Capabilities |
| 16 Apr 2025 | BitNet b1.58 2B4T technical report | 45.3 | TruthfulQA (10-shot; MC2) (Table 1, Table 3) | Hallucination |
| 3 Mar 2025 | Phi-4 Mini technical report | 66.4 | TruthfulQA (10-Shot, MC2) (Table 7) | Hallucination |
| 31 Dec 2024 | OLMo 2 32B Instruct technical report | 73.2 | TruthfulQA, 6-shot MC2 (Tables 7 and 16) | Capabilities |
| 30 Aug 2024 | phi-3-small 7.4B technical report | 70.2 | TruthfulQA (10-Shot, MC2) (Section 3 table) | Hallucination |
| 30 Aug 2024 | phi-3-mini 3.8B technical report | 65 | TruthfulQA (10-Shot, MC2) (Section 3 table) | Hallucination |
| 30 Aug 2024 | phi-3-medium 14B technical report | 75.1 | TruthfulQA (10-Shot, MC2) (Section 3 table) | Hallucination |
| 30 Aug 2024 | Phi-3.5-MoE technical report | 77.5 | TruthfulQA (10-shot, MC2) (Table 3) | Hallucination |
| 30 Aug 2024 | Phi-3.5-mini technical report | 64 | TruthfulQA (10-shot, MC2) (Table 3) | Hallucination |
| 31 Jul 2024 | Gemma 2 9B technical report | 50.3 | TruthfulQA, MC2Acc, IT (Table 18) | Hallucination |
| 31 Jul 2024 | Gemma 2 27B technical report | 51.6 | TruthfulQA, MC2Acc, IT (Table 18) | Hallucination |
| 13 Mar 2024 | Gemma 7B technical report | 31.8 | TruthfulQA, MC2, IT (Table 8). HuggingFace H6 run of the PT model (Table 7): 44.8 | Hallucination |
| 13 Mar 2024 | Gemma 2B technical report | 44.8 | TruthfulQA, MC2, IT (Table 8) | Hallucination |
| 19 Jul 2023 | Llama 2-7B technical report | 33.3 | TruthfulQA, % truthful and informative generations, pretrained (Table 11). Llama 2-Chat = 57.04 (Table 14) | Safety |
| 19 Jul 2023 | Llama 2-70B technical report | 50.2 | TruthfulQA, % truthful and informative generations, pretrained (Table 11). Llama 2-Chat = 64.14 (Table 14) | Safety |
| 19 Jul 2023 | Llama 2-34B technical report | 43.5 | TruthfulQA, % truthful and informative generations, pretrained (Table 11). Llama 2-Chat = 67.2 (Table 14) | Safety |
| 19 Jul 2023 | Llama 2-13B technical report | 41.9 | TruthfulQA, % truthful and informative generations, pretrained (Table 11). Llama 2-Chat = 62.18 (Table 14) | Safety |
| 18 Jul 2023 | Llama 2-7B model card | 33.3 | TruthfulQA, % truthful and informative, pretrained (automatic safety benchmarks table). Llama-2-Chat = 57.04 | Safety |
| 18 Jul 2023 | Llama 2-70B model card | 50.2 | TruthfulQA, % truthful and informative, pretrained (automatic safety benchmarks table). Llama-2-Chat = 64.14 | Safety |
| 18 Jul 2023 | Llama 2-13B model card | 41.9 | TruthfulQA, % truthful and informative, pretrained (automatic safety benchmarks table). Llama-2-Chat = 62.18 | Safety |
| 11 Jul 2023 | Claude 2 model card | 69 | TruthfulQA with free-form sampling, answers mapped to multiple choice options by the helpful-only model, score 0.69 (3.3, Figure 4) | Hallucination |
| 14 Mar 2023 | GPT-4 technical report | 60 | TruthfulQA mc1, post-trained GPT-4, around 60% stated in System Card text (3.1), Figure 7 chart has no printed numbers | Hallucination |
More in Factuality
- SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. Qwen3.7-Plus 81.7%
- HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy.
Keye-VL-2.0-30B-A3B 73.5% - SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. Gemini 3.1 Pro 77.5%
- GaRAGe In use Models write long-form RAG answers to 2,366 questions with human-annotated grounding passages. Scoring checks whether answers use the relevant passages.
Nemotron 3.5 Bolt E4B 40.7% - Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. Kimi K3.1 Pro 8.30%
- RAGTruth In use A word-level annotated corpus of hallucinations in retrieval-augmented generation answers. It is used to train and evaluate hallucination detection and RAG faithfulness. Nemotron 3.5 Bolt E4B 77.9%