Factuality
We track 70 benchmarks in factuality, or 77 if you count each version. They run from everyday level to expert level.
Google DeepMind held the top factuality score 36% of the contested time
That's about 17 of the 46 months since January 2023, averaged over the benchmarks below. The easiest benchmarks are at the top. Versions of one test share a row, with a dark tick at each new version. We stop the clock, and fade the rest of the row, when the best score passes 85%, or at the newest score if 45 days go by without another.
Anthropic Mistral AI Google DeepMind Moonshot AI OpenAI Qwen ByteDance Seed Z.ai StepFun NVIDIA Kuaishou Meta Perplexity Amazon DeepSeek xAI Tencent Baidu
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Share of contested time at number one
Google DeepMind 36% OpenAI 20% 
ByteDance Seed 8% 
Qwen 8% Anthropic 7% 
Baidu 4% 
NVIDIA 4% 
Meta 4% 
Tencent 2% 
Mistral AI 2% 
Kuaishou 2% Z.ai 1% Closed out: 6 benchmarks passed 85%
Anthropic 2 Baidu 1 Moonshot AI 1 Google DeepMind 1 
Perplexity 1
Baidu 1 Moonshot AI 1 Google DeepMind 1 Perplexity 1 - Mar 2026 BullshitBench V2 Claude Sonnet 4.6
- Feb 2026 Chinese SimpleQA ERNIE 5.0
- Feb 2026 BullshitBench Claude Opus 4.5
- Jul 2025 Vectara Hallucination Leaderboard (HHEM) Kimi K2 Jul 2025
- Feb 2025 FACTS Grounding Gemini 2.0 Flash Feb 2025
- Jan 2025 SimpleQA Sonar Pro
Scores come from each benchmark’s own leaderboard, from Epoch AI’s runs and from labs’ reports on their own models. We date Epoch runs by the model’s release and lab reports by when they came out. Labs pick their own settings, so a lab’s claim can beat an independent run of the same model. Turn off lab claims to see independent runs only.
Anthropic Mistral AI Google DeepMind OpenAI Meta Qwen
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Share of contested time at number one, independent runs only
Google DeepMind 60% OpenAI 20% Anthropic 10% Meta 5% Mistral AI 3% Qwen 2% Only each benchmark’s own leaderboard and Epoch AI’s runs, dated by the model’s release. We leave out benchmarks that only labs report.
From everyday to expert level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Everyday
Most adults, without training
- Vectara Hallucination Leaderboard (HHEM)
- BullshitBench V2
- BullshitBench
- Vectara Hallucination Leaderboard (2025 dataset)
- Phare
- LLM Confabulation Leaderboard
- HallusionBench
- TruthfulQA
- FaithJudge
- POPE
- FActScore
- PersonQA
- False-premise questions (Anthropic)
- Honesty human feedback win rate (Anthropic)
- LLM-AggreFact
- LongFact-Concepts
- LongFact-Objects
- Microsoft RAI benchmark: Ungroundedness
- Factuality Heavy prompts
- AbstentionBench
- Answer Factuality (Perplexity)
- BirthdayFacts
- GaRAGe
- Hallucinations on production-representative prompts
- HaluEval
- Kitab
- Open Ended Questions (hallucination)
- RAGTruth
- CounterFactQA (Qwen)
- Enterprise RAG evaluation (Cohere)
- Faithfulness (Moonshot)
- Grounding with citations, sentence unfactual rate
- Long-context grounding, sentence unfactual rate
- Microsoft RAI benchmark: Grounding
- Multi-factual (Anthropic)
- RGB
- SuperCLUE-RAG
- Agentic Abstention
- AMBER
- Closed-domain hallucination evaluation
- CRAG
- HaluMem
- Input hallucinations (Anthropic)
- Internal adversarially-designed factuality evaluations
- Missing-context hallucinations (Anthropic)
University
Undergraduates and entry-level professionals
- SimpleQA Verified
- SimpleQA
- FACTS Grounding
- Chinese SimpleQA
- FACTS Parametric
- FACTS Benchmark Suite
- SimpleVQA
- FACTS Multimodal
- FACTS Search
- FACTS Search V2
- FACTS Parametric V2
- FACTS Grounding v2
- Video SimpleQA
- CharXiv Missing Images
- CodeSimpleQA
- MMLU calibration (expected calibration error)
- HaluBench
Expert
Specialists with years of training, often over hours of work
Another 6 have no level. Levels don’t apply to arenas, safety evaluations or composite indices.
By kind
Short factual questions
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| SimpleQA Verified 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. | University | Graded by a model | 77.5% | In use |
| SimpleQA Models answer 4,326 short fact questions that each have one indisputable answer. A grader model marks each reply correct, incorrect, or not attempted. | University | Checked automatically |  97.1% | Fading |
| Chinese SimpleQA 3,000 short Chinese factual questions across six topics. Graded correct, incorrect or not attempted. | University | Graded by a model | 86% | Fading |
| FACTS Parametric Models answer closed-book factoid questions from their own knowledge, without tools. A judge model grades each answer. | University | Graded by a model | 83.1% | Fading |
| FACTS Benchmark Suite Models are scored on four factuality tests: Parametric, Search, Multimodal and Grounding v2. The FACTS Score averages accuracy over public and private sets of all four. | University | Graded by a model | 75.4% | Fading |
| SimpleVQA Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. | University | Checked automatically | 81.7% | In use |
| TruthfulQA Models answer 817 questions across 38 categories that are written to elicit common misconceptions. Scoring checks for truthful answers, often in multiple-choice form. | Everyday | Checked automatically |  88% | Fading |
| FACTS Multimodal Models answer questions about input images and must combine what they see with world knowledge. A judge checks coverage and contradictions. | University | Graded by a model | 50.7% | Not in cards |
| AA-Omniscience 41-topic closed-book knowledge test. Net score rewards correct answers, penalizes wrong ones and gives zero for abstentions. | Expert | Checked automatically | 70.8% | In use |
| FACTS Search Models answer information-seeking questions with a web search tool (the Brave Search API) and must combine what they find correctly. | University | Graded by a model | 85.6% | Not in cards |
| FACTS Search V2 Models answer information-seeking questions with a search tool and must combine what they find correctly. Second version of FACTS Search on Kaggle. | University | Graded by a model | 89.9% | Not in cards |
| FACTS Parametric V2 Models answer closed-book factoid questions from their own knowledge, without tools. A judge model grades each answer. | University | Graded by a model | 83.8% | Not in cards |
| 100Q-Hard (Anthropic) Models answer hard, obscure factual questions. Scored on correct, incorrect and declined answers. | Expert | Graded by a model | 60.1% | System cards only |
| Video SimpleQA Video models answer multi-hop fact-seeking questions that need knowledge beyond the video and evidence from its time segments. Short definitive answers are graded for factual correctness. | University | Graded by a model | 76.4% | Fading |
| CodeSimpleQA Models answer short bilingual fact questions about programming concepts and implementations across languages and computer science domains. | University | Graded by a model | 58% | Fading |
| MMLU calibration (expected calibration error) Models answer an MMLU subset and their answer probabilities are compared with accuracy. Scored as expected calibration error. | University | Checked automatically | 0.07 | System cards only |
| AA-Omniscience v1.1 Models answer 3,000 knowledge questions in six domains without tools and may abstain. The index adds correct answers and subtracts wrong ones. | Expert | Graded by a model | Not in cards |
Faithful to sources
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) Models summarize short documents. The HHEM-2.1-Open classifier flags summaries that add unsupported facts, giving a hallucination rate. | Everyday | Graded by a model | 27.8% | System cards only |
| Vectara Hallucination Leaderboard (2025 dataset) Models summarize over 7,700 private articles of up to 32K tokens using only facts in the text. Vectara's commercial HHEM model flags hallucinated summaries. | Everyday | Graded by a model | 3.10% | Not in cards |
| LLM Confabulation Leaderboard Models answer misleading questions about recent articles whose answers are not in the text. The board ranks confabulation rate against non-response rate. | Everyday | Graded by a model | 10.3% | Not in cards |
| FACTS Grounding Long-form responses must be fully grounded in a provided document. Judged by LLM panels for grounding and eligibility. | University | Graded by a model | 88.5% | Dropped |
| FaithJudge Models answer retrieval-augmented tasks from given passages. An LLM judge guided by human-annotated examples flags unfaithful answers, giving a hallucination rate. | Everyday | Graded by a model | 6.26% | System cards only |
| FACTS Grounding v2 Models write long answers from a supplied document of up to 32k tokens. Judge models check that each answer stays faithful to the document. | University | Graded by a model | 80.1% | Not in cards |
| LLM-AggreFact Models judge whether claims are supported by a grounding document across 11 pooled datasets, such as RAGTruth and ExpertQA. Scored by balanced accuracy. | Everyday | Checked automatically | 77.2% | Not in cards |
| Microsoft RAI benchmark: Ungroundedness Models answer in GPT-4 simulated multi-turn conversations. A judge rates how ungrounded each answer is in the given prompt, from 0 to 4. | Everyday | Graded by a model | 0.21 | System cards only |
| Answer Factuality (Perplexity) Models answer questions from search results. Graders check that answers use facts grounded in the results and resolve conflicting or missing information. | Everyday | Graded by a model | 85.1% | Dropped |
| GaRAGe Models write long-form RAG answers to 2,366 questions with human-annotated grounding passages. Scoring checks whether answers use the relevant passages. | Everyday | Graded by a model | 40.7% | In use |
| Enterprise RAG evaluation (Cohere) Models answer workplace policy and technical questions over long documents, some unanswerable. Llama Index Correctness grades answers from 1 to 5. | Everyday | Graded by a model |  4.73 | System cards only |
| Grounding with citations, sentence unfactual rate Models answer multi-snippet tasks with inline citations. An autorater reports the share of sentences not supported by the sources. | Everyday | Graded by a model | 8.20% | System cards only |
| Long-context grounding, sentence unfactual rate Models answer from long documents. An autorater reports the share of sentences not supported by the documents. | Everyday | Graded by a model | 6.40% | System cards only |
| Microsoft RAI benchmark: Grounding Models answer in GPT-4o simulated multi-turn conversations. A judge scores from 0 to 5 how fully each answer is grounded in the given prompt. | Everyday | Graded by a model | 4.62 | System cards only |
| RGB Models answer English and Chinese news questions from retrieved passages, testing noise robustness, negative rejection, information integration, and counterfactual robustness. Scored by accuracy. | Everyday | Checked automatically | Dropped | |
| SuperCLUE-RAG Chinese models answer questions twice, first without documents and then with supplied documents. A judge model compares the answers to rate how well the system uses retrieved text. | Everyday | Graded by a model | 66.7 | Not in cards |
| Closed-domain hallucination evaluation Models answer using only provided context. Scored on avoiding invented details outside that context. | Everyday | Graded by a model | System cards only | |
| CRAG Retrieval-augmented systems answer 4,409 factual questions across finance, sports, music, movies and open domain, using mock web and knowledge graph search APIs. Hallucinated answers are penalized. | Everyday | Graded by a model | Not in cards |
Long-form factuality
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| LongFact Models write long answers to 2,280 fact-seeking prompts on 38 topics. A search-backed checker verifies each claim, and cards report the hallucination rate. | Graded by a model | 0.70% | Fading | |
| FActScore Models write biographies of people, and each output is split into atomic facts. The score is the share of facts that a knowledge source supports. | Everyday | Graded by a model | 1.00% | Fading |
| LongFact-Concepts Models write detailed answers to LongFact prompts about broad concepts. A fact checker scores the claim-level error rate. | Everyday | Graded by a model | 0.90% | System cards only |
| LongFact-Objects Models write detailed answers to LongFact prompts about specific objects such as people or places. A fact checker scores the claim-level error rate. | Everyday | Graded by a model | 1.40% | System cards only |
| Kitab Models list books by a given author that meet extra constraints, such as publication dates or title words. Scored by precision and constraint satisfaction. | Everyday | Checked automatically | Dropped | |
| Open Ended Questions (hallucination) Models write open-ended factual text such as biographies, checked against Wikipedia. Scored as average number of incorrect statements. | Everyday | Graded by a model | 0.78 | System cards only |
| Multi-factual (Anthropic) Models answer questions that each combine several closed-ended sub-questions on one topic. Answers are graded correct, incorrect or unsure against references. | Everyday | Graded by a model | 62.8% | System cards only |
Hallucination rates
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| BullshitBench V2 Models answer 100 nonsense questions across software, finance, legal, medical and physics, built with 13 techniques. A three-judge panel scores clear pushback. | Everyday | Graded by a model | 95% | Not in cards |
| BullshitBench Models get prompts built on nonsensical premises. A three-judge model panel grades whether each answer pushes back clearly, challenges partly or accepts the nonsense. | Everyday | Graded by a model | 96.4% | Not in cards |
| Phare Models are probed in English, French and Spanish for hallucination, harmful misguidance, stereotypes and jailbreaks. Hallucination tasks cover factuality, misinformation, debunking and tool use. | Everyday | Graded by a model | 83.2% | Not in cards |
| HallusionBench Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy. | Everyday | Checked automatically | 73.5% | In use |
| SimpleQA Hallucination Rate Models answer 4,326 short fact-seeking questions. This metric is the share of questions answered incorrectly, so declining to answer is not penalized. | Checked automatically | 7.90% | Dropped | |
| POPE Vision language models answer yes or no questions about whether objects appear in COCO images. Scored by accuracy or F1 on object hallucination. | Everyday | Checked automatically |  90.7% | Fading |
| PersonQA Models answer questions about publicly available facts on people. Scored on accuracy, with hallucination rate reported alongside. | Everyday | Graded by a model | 94.3% | System cards only |
| False-premise questions (Anthropic) Models answer questions built on false premises. Measures how often they accept the premise instead of correcting it. | Everyday | Graded by a model | 1.20% | System cards only |
| Honesty human feedback win rate (Anthropic) Crowdworkers try to make models say something false and pick the more honest reply in head-to-head chats. Score is the win rate against a baseline. | Everyday | Ranked by votes | 69% | System cards only |
| OpenAI user-flagged hallucination eval Production conversations users flagged for factual errors. Measures the rate of incorrect claims. | Graded by a model | 46.1% | System cards only | |
| Factuality Heavy prompts Models answer prompts representative of factuality-heavy ChatGPT conversations. A grader with web access counts responses and claims with factual errors. | Everyday | Graded by a model | 5.40% | System cards only |
| HealthBench Hard Hallucinations Models respond to difficult health conversations that 2 or more physicians validated. The score is the hallucination rate, so lower is better. | Graded by a model | 1.60% | Dropped | |
| High Stakes factuality prompts Models answer difficult medical, legal and financial prompts. A grader with web access counts responses and claims with factual errors. | Expert | Graded by a model | 4.10% | System cards only |
| HLE Calibration Models give a confidence score with each HLE answer. The benchmark reports RMS calibration error and AUROC between confidence and correctness. | Expert | Checked automatically | 23.4 | System cards only |
| AbstentionBench Models face underspecified, ill-posed or unanswerable questions. Scored on recall of appropriate abstention. | Everyday | Graded by a model | 59.1% | System cards only |
| BirthdayFacts Models are asked for people's birthdays. Scored on how often they give a wrong birthday. | Everyday | Checked automatically | 24% | System cards only |
| CharXiv Missing Images Models answer CharXiv chart questions with the image removed. The benchmark reports how often they answer incorrectly instead of declining. | University | Checked automatically | 12.3% | System cards only |
| Hallucinations on production-representative prompts Models answer prompts representative of ChatGPT production traffic. A grader with web access scores incorrect claims and responses with major errors. | Everyday | Graded by a model | 3.10% | System cards only |
| HaluEval Models judge whether answers, dialogue turns or summaries contain hallucinations. Hallucinated samples were generated by ChatGPT from seed datasets such as HotpotQA and then filtered. | Everyday | Checked automatically | 62.2% | Fading |
| HealthBench-Hallu Models answer HealthBench-style medical prompts. A pipeline splits each answer into medical claims and checks them against medical evidence. The score is a weighted hallucination rate. | Expert | Graded by a model |  3.50% | Fading |
| Kimi Hallu Bench Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. | Expert | Graded by a model | 8.30% | In use |
| RAGTruth A word-level annotated corpus of hallucinations in retrieval-augmented generation answers. It is used to train and evaluate hallucination detection and RAG faithfulness. | Everyday | Graded by people | 77.9% | In use |
| xAI Factuality (Hallucination) eval Internal measure of unsupported or fabricated claims in single responses to information-seeking queries. | Graded by a model | 1.70% | System cards only | |
| CounterFactQA (Qwen) Models get counterfactual questions and must recognize that the premise is false instead of giving a hallucinated answer. | Everyday | Graded by a model | 68.1% | System cards only |
| Faithfulness (Moonshot) A fact checker verifies each model response for factual hallucinations. The score is one minus the hallucination rate. | Everyday | Graded by a model | 85.5% | System cards only |
| Meta internal agent hallucination evaluation Agents do long-context assistant and professional tasks with tools. The benchmark reports the share of verifiable claims that are hallucinated. | Expert | Graded by a model | 5.74% | System cards only |
| Agentic Abstention Agents work in web shopping, terminal and search question tasks where some goals cannot be met. They must abstain at the right turn instead of acting on. | Everyday | Checked automatically | Not in cards | |
| AMBER Multimodal models are checked for object, attribute and relation hallucinations in generative and discriminative tasks, scored without an LLM judge. | Everyday | Checked automatically | Not in cards | |
| HaluBench Models judge whether an answer is faithful to its context across about 15,000 labeled context, question and answer triplets. It tests hallucination detectors and LLM judges. | University | Checked automatically | Not in cards | |
| HaluMem Agent memory systems are tested for hallucinations at each operation, memory extraction, updating and question answering, rather than only end to end. | Everyday | Checked automatically | Not in cards | |
| Input hallucinations (Anthropic) Models answer 500 prompts that need an unavailable tool and 500 prompts that cite missing context. Score is the non-hallucination rate. | Everyday | Graded by a model | System cards only | |
| Internal adversarially-designed factuality evaluations Models answer adversarial questions in nine categories such as history, math and code. Scored on agreement with human ideal responses. | Everyday | Graded by a model | System cards only | |
| Missing-context hallucinations (Anthropic) Models get prompts that request tools they lack or cite missing material. Score is the non-hallucination rate on each 500-prompt set. | Everyday | Graded by a model | System cards only | |
| OpenAI performance in cases flagged by users Model answers de-identified ChatGPT prompts that users flagged for factual errors. Scored on any error and on repeating the flagged error. | Graded by a model | System cards only | ||
| SWE agentic fabrication eval A grounding autorater checks whether claims made by coding agents on long SWE tasks are supported by tool outputs. | Expert | Graded by a model | System cards only |