Long context
We track 66 benchmarks in long context, or 73 if you count each version. They run from everyday level to expert level.
OpenAI held the top long context score 37% of the contested time
That's about 17 of the 46 months since January 2023, averaged over the benchmarks below. The easiest benchmarks are at the top. Versions of one test share a row, with a dark tick at each new version. We stop the clock, and fade the rest of the row, when the best score passes 85%, or at the newest score if 45 days go by without another.
Z.ai Microsoft Qwen Mistral AI NVIDIA OpenAI Google DeepMind MiniMax LG AI Research DeepSeek Meta Anthropic Cohere ByteDance Seed Tencent Moonshot AI StepFun Reflection
Everyday Most adults, without training
School Students, from grade school to high school
University Undergraduates and entry-level professionals
Cohere Command A Expert Specialists with years of training, often over hours of work
Share of contested time at number one
OpenAI 37% 
Google DeepMind 17% Z.ai 15% 
MiniMax 9% Anthropic 6% 
DeepSeek 3% 
Tencent 3% 
ByteDance Seed 3% 
Meta 2% 
Qwen 2% 
Mistral AI 2% Reflection 1% Moonshot AI 1%
Google DeepMind 17% Z.ai 15% MiniMax 9% Anthropic 6% DeepSeek 3% Tencent 3% ByteDance Seed 3% Meta 2% Qwen 2% Mistral AI 2% Reflection 1% Moonshot AI 1% Closed out: 10 benchmarks passed 85%
Anthropic 3 OpenAI 2 
StepFun 1 
LG AI Research 1 Qwen 1 
Microsoft 1 Z.ai 1
StepFun 1 LG AI Research 1 Qwen 1 Microsoft 1 Z.ai 1 - Sep 2026 AA-LCR 1.1 Step 5 Preview
- Aug 2026 OpenAI MRCR K-EXAONE 2.0
- Jul 2026 OpenAI MRCR v2 GPT-5.6 Terra
- May 2026 GraphWalks Claude Opus 4.8
- May 2026 MRCR v2 Qwen3.7-Max
- Apr 2025 Fiction.LiveBench o3
- Aug 2024 RULER Phi-3.5-MoE
- Jun 2024 RepoQA Claude 3 Opus
- Mar 2024 QuALITY Claude 3 Opus
- Jan 2024 Needle In A Haystack GLM-4
Scores come from each benchmark’s own leaderboard, from Epoch AI’s runs and from labs’ reports on their own models. We date Epoch runs by the model’s release and lab reports by when they came out. Labs pick their own settings, so a lab’s claim can beat an independent run of the same model. Turn off lab claims to see independent runs only.
OpenAI Google DeepMind Anthropic DeepSeek
Everyday Most adults, without training
University Undergraduates and entry-level professionals
Expert Specialists with years of training, often over hours of work
Share of contested time at number one, independent runs only
OpenAI 49% Google DeepMind 27% DeepSeek 13% Anthropic 10%
Google DeepMind 27% DeepSeek 13% Anthropic 10% Only each benchmark’s own leaderboard and Epoch AI’s runs, dated by the model’s release. We leave out benchmarks that only labs report.
From everyday to expert level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Everyday
Most adults, without training
- Fiction.LiveBench
- RULER
- MRCR v2
- Document Arena
- CL-bench Life
- OpenAI MRCR
- OpenAI MRCR v2
- MRCR (Michelangelo, original)
- Needle In A Haystack
- LOFT
- MMLongBench
- LongBench-Chat
- FlenQA
- InfiniteBench/En.MC
- ZeroSCROLLS/QuALITY
- InfiniteBench/En.QA
- Passkey Retrieval
- BEAM
- LongMemEval
- LV-Eval
- MemoryBench (Meta)
- VCSUM
- Long Context (MiniMax in-house)
- PenguinScrolls
- Agent Memory Leaderboard
- AMemGym
- BABILong
- ChatRAG Bench
- HABIT-Bench
- KnowMe-Bench
- LoCoMo
- MemEye
- MemLens
- MemoryAgentBench
- MemTrapBench
- MM-NIAH
- MR-NIAH (Multi-Round Needles-In-A-Haystack)
- NeedleBench
- NeedleBench V2
- NoLiMa
- RULERv2
University
Undergraduates and entry-level professionals
Expert
Specialists with years of training, often over hours of work
By kind
Finding facts in long inputs
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| RULER Synthetic retrieval, multi-hop tracing, aggregation and QA tasks at configurable lengths. Measures effective context size. | Everyday | Checked automatically |  99% | In use |
| MRCR v2 Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. | Everyday | Checked automatically | 98.5% | In use |
| RepoQA Models find a function in a long code repository from a description of what it does, across Python, C++, Rust, Java and TypeScript. | University | Checked automatically | 90.6% | Dropped |
| OpenAI MRCR Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio, in context bins from 4K to 1M tokens. | Everyday | Checked automatically | 94.4% | Dropped |
| OpenAI MRCR v2 Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. | Everyday | Checked automatically | 94.4% | In use |
| MRCR (Michelangelo, original) Models retrieve the n-th of several similar requests hidden in a long synthetic conversation. Original Michelangelo version before MRCR v2. | Everyday | Checked automatically | 94.5% | Dropped |
| Needle In A Haystack Models retrieve a planted fact from long filler text at varied context lengths and insertion depths. Scored by retrieval accuracy. | Everyday | Checked automatically | 100% | Dropped |
| LOFT Models solve retrieval, RAG, SQL, and many-shot learning tasks with whole corpora of up to one million tokens in context. Scored per task. | Everyday | Checked automatically | 87% | Dropped |
| MMLongBench Vision-language models solve 13,331 long-context examples in five task types, such as visual RAG, needle search and many-shot in-context learning. Each example comes at five lengths from 8K to 128K tokens. | Everyday | Checked automatically | 78.3% | Fading |
| Passkey Retrieval Models find a random passkey hidden at a random position in a long filler prompt. Scored by retrieval accuracy. | Everyday | Checked automatically | 100% | Dropped |
| LongMemEval Chat assistants answer 500 questions about long multi-session chat histories. The questions test five long-term memory abilities. | Everyday | Graded by a model | 69.9% | In use |
| MemoryBench (Meta) Models recall key facts from multi-turn conversations of 64K to 100K tokens that include distractor chats and turns that conflict with or update the facts. An LLM judge scores answers on weighted rubrics. | Everyday | Graded by a model | In use | |
| ChatRAG Bench Models answer multi-turn conversational questions over documents or retrieved context. The suite is built from 10 existing datasets such as Doc2Dial, QuAC, QReCC and CoQA. | Everyday | Checked automatically | System cards only | |
| LoCoMo Models answer questions, summarize events and continue dialogue over very long multi-session chats of about 300 turns and 9K tokens, up to 35 sessions. Answers are scored against references. | Everyday | Checked automatically | Not in cards | |
| MemEye Multimodal agent memory systems answer questions that need fine visual evidence kept from earlier sessions, which captions or text traces alone cannot supply. | Everyday | Checked automatically | Not in cards | |
| MemLens Vision-language models and memory agents answer 789 questions over multimodal multi-session conversations, covering extraction, temporal reasoning, knowledge updates and refusal at four context lengths. | Everyday | Checked automatically | Not in cards | |
| MM-NIAH Vision-language models answer questions about long documents of interleaved images and text. Tasks are retrieval, counting and reasoning over needles hidden in the document. | Everyday | Checked automatically | Not in cards | |
| MR-NIAH (Multi-Round Needles-In-A-Haystack) Models must repeat one earlier answer from a long multi-turn dialogue history of 2K to 1M tokens. Scored by adjusted recall of key parts. | Everyday | Graded by a model | System cards only | |
| NeedleBench Models retrieve and reason over one or several facts inserted at different depths in bilingual long texts of set lengths. | Everyday | Checked automatically | System cards only | |
| NeedleBench V2 Models retrieve one or several needles and reason over fictional multi-needle facts in English and Chinese haystacks from 4K to 1M tokens. The overall score is the plain average of three tasks. | Everyday | Checked automatically | Not in cards | |
| NoLiMa Models find a needle in a long haystack when the question and the needle share almost no words, so they must infer a latent link. Accuracy is compared across context lengths. | Everyday | Checked automatically | Not in cards | |
| RULERv2 Models solve synthetic multi-key and multi-value needle tasks and QA tasks at basic, easy, medium and hard levels. Contexts run from 8K to 1M tokens, and accuracy is averaged across lengths. | Everyday | Checked automatically | Not in cards |
Reasoning over long documents
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| LongBench v2 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. | Expert | Checked automatically | 66.9% | In use |
| AA-LCR (Artificial Analysis Long Context Reasoning) Questions requiring reasoning across multiple real documents totaling about 100K tokens. Answers judged by an LLM. | University | Checked automatically | 80% | In use |
| Fiction.LiveBench Models answer 36 questions about 30 Fiction.live stories that need inference about plot and characters. Each story is tested at several context lengths. | Everyday | Checked automatically | 97.2% | Not in cards |
| GraphWalks A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. | University | Checked automatically | 100% | In use |
| Document Arena People upload their own PDFs with a question, compare answers from two anonymous models, and vote for the better one. Votes become Bradley-Terry ratings. | Everyday | Ranked by votes | 1,499 | Not in cards |
| LOCA-Bench Agentic tasks whose environment state is scaled to grow context length while task semantics stay fixed. Success rate versus context length. | University | Checked automatically | 21.3% | In use |
| QuALITY Models answer multiple-choice questions about English passages that average about 5,000 tokens. Scoring is accuracy. | School | Checked automatically | 90.5% | Dropped |
| AA-LCR 1.1 Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. | University | Checked automatically | 88.3% | In use |
| L-Eval Models answer over 2,000 queries on 508 long documents of 3k to 200k tokens across 20 subtasks. Scoring uses exact match and judge-model ratings. | University | Checked automatically | 76.4% | Dropped |
| LongBench-Chat Models follow real user instructions over contexts of 10K to 100K tokens. GPT-4 scores each answer from 1 to 10 against references. | Everyday | Graded by a model | 8.76 | Dropped |
| BrowseComp Long Context 128k Models answer BrowseComp questions from a 128k-token context of the needed web pages mixed with related distractor pages. Scored by accuracy. | Expert | Checked automatically | 90% | Dropped |
| Ko-LongBench (LG) Models answer questions over long Korean contexts. Tasks cover document QA, story and dialogue history understanding, in-context learning, structured QA and RAG. | University | Checked automatically | 89.6% | Fading |
| FlenQA Models answer reasoning questions whose key facts are padded with filler text to different lengths. Accuracy shows how input length affects reasoning. | Everyday | Checked automatically | 97.9% | Dropped |
| InfiniteBench/En.MC Models answer 229 multiple choice questions about long fake books with altered entity names, averaging about 184K tokens. Scoring is accuracy. | Everyday | Checked automatically | 83.4% | Dropped |
| LongBench Models work through 21 English and Chinese long-input datasets covering QA, summarization, few-shot learning, synthetic tasks, and code. Each dataset has its own metric. | University | Checked automatically | 81.1% | Dropped |
| LongCodeBench Coding model answers repository QA and fixes GitHub issues with context from 32K to 1M tokens. Accuracy and resolve rate. | Expert | Checked automatically |  84% | Fading |
| Qasper Models answer 5,049 questions about 1,585 NLP research papers from the full paper text. Answers are scored with token F1. | University | Checked automatically | 41.9% | Dropped |
| SQuALITY Models write question-focused summaries of public-domain short stories. Outputs are scored against human-written reference summaries with ROUGE. | School | Checked automatically | 24.3% | Dropped |
| ZeroSCROLLS/QuALITY Models answer QuALITY multiple-choice questions about long stories zero-shot, as one task of the ZeroSCROLLS suite. Scoring is accuracy. | Everyday | Checked automatically | 95.2% | Dropped |
| CorpusQA 1M Models answer computation-heavy questions whose evidence is spread across hundreds of finance, education, and real estate documents in a 1M-token context. Scored by accuracy. | University | Checked automatically | 62% | Fading |
| InfiniteBench/En.QA Models answer free-form questions about novels longer than 100K tokens. Scored by F1 against reference answers. | Everyday | Checked automatically | 36.7 | System cards only |
| HELMET Models handle long-context tasks in seven groups including recall, RAG, re-ranking, many-shot learning, long QA and summarization, at set input lengths. | University | Graded by a model | 58.3% | Dropped |
| LV-Eval Models answer single-hop and multi-hop questions over contexts at five length levels from 16K to 256K tokens, with confusing facts inserted. | Everyday | Checked automatically | 49.8% | Dropped |
| VCSUM Models summarize transcripts of 239 real Chinese meetings that total over 230 hours. Scored with ROUGE against human-written summaries. | Everyday | Checked automatically | 17.3 | Dropped |
| CorpusQA Corpus-level questions over hundreds of documents up to 10M tokens needing aggregation and statistics. Programmatically guaranteed answers, accuracy. | University | Checked automatically | 82% | System cards only |
| Long Context (MiniMax in-house) Models translate, summarize and analyze long documents from real user requests. | Everyday | Graded by a model | 93.8% | System cards only |
| PenguinScrolls Models answer user-style questions over long natural documents such as books, financial reports and legal papers, up to 128K tokens. Tasks cover extraction, localization, qualitative analysis and numerical reasoning. | Everyday | Graded by a model | 87.7% | System cards only |
| AA-LCR Extended Models answer questions that need reasoning across about 100k tokens of documents. Mercor adds 89 new expert-written tasks to the 100 public ones and reports both scores. | Expert | Checked automatically | Not in cards | |
| BABILong Models answer bAbI reasoning questions whose supporting facts are scattered through very long distractor text, at lengths up to millions of tokens. | Everyday | Checked automatically | Not in cards | |
| BrowseComp Long Context 256k Models answer BrowseComp questions from a context of 128K to 256K tokens built from the needed web pages and related distractor pages. Scored by accuracy. | Expert | Checked automatically | Not in cards | |
| CorporateBench Agents answer questions over the email archives of four synthetic companies, up to 232,692 documents. Answers are checked against each company's ground-truth knowledge graph. | Expert | Checked automatically | Not in cards | |
| SCROLLS Models do seven long-text tasks such as summarization, question answering and natural language inference over naturally long documents. | University | Checked automatically | Not in cards | |
| Summary of a Haystack Models read a synthetic haystack of conversation or news documents where specific insights repeat, then write a query-focused summary that cites its sources. An automatic evaluation scores insight coverage and citation accuracy. | University | Graded by a model | Not in cards | |
| WildGraphBench GraphRAG systems answer 1,100 questions over long, mixed reference documents behind Wikipedia articles in 12 topics, from single facts to section summaries. | University | Graded by a model | Not in cards |
Learning from the context
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| CL-bench Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. | University | Graded by a model | 27.9% | In use |
| CL-bench Life Models solve 405 tasks that require learning from messy real-life context such as group chats. A model judge checks rubrics to give a solve rate. | Everyday | Graded by a model | 22.2% | Fading |
| Continual Learning Exam Agents work through 120 professional tasks, each a ladder of five levels that must be passed in order. Later levels reuse files, tools and lessons from earlier ones. An evaluation agent grades against expert rubrics. | Expert | Graded by a model | 47.7% | Not in cards |
| MTOB Models learn to translate between English and Kalamang, a language with under 200 speakers, from one grammar book. Scored by chrF. | Expert | Checked automatically | 54.0 | Dropped |
| BEAM Models answer probing questions about coherent multi-session user conversations of 128K, 500K, 1M and 10M tokens. Questions test ten memory abilities such as knowledge updates, temporal reasoning and contradiction resolution. | Everyday | Graded by a model | 65.1% | In use |
| CLIF Bench Models learn from a provided context while they follow instructions that combine several complex skills. | Expert | Checked automatically | 52.4% | System cards only |
| Agent Memory Leaderboard Memory systems for agents are evaluated on storing, retrieving and using long-term memory across textual, coding and multimodal tracks in one reproducible framework. | Everyday | Checked automatically | Not in cards | |
| AMemGym Assistants hold long simulated conversations with users whose details and preferences change over time. Simulated users react on policy, and the test checks whether the assistant remembers and applies the right facts. | Everyday | Graded by a model | Not in cards | |
| Continual Learning Bench AI systems work through sequences of related task instances in six domains and should get better as they go. A gain score subtracts each system's own stateless baseline to isolate learning. | Expert | Checked automatically | Not in cards | |
| ContinualSkillBench Agents work through five domains of 100 linked subtasks of rising difficulty, testing whether skill libraries they build in context actually help later tasks. | Expert | Checked automatically | Not in cards | |
| EvoArena Agents work in environments whose knowledge, tools and task conditions change over time. The suite tests whether agent memory keeps up with the changes. | Expert | Checked automatically | Not in cards | |
| HABIT-Bench Agent memory systems must infer user habits from weak evidence spread across many past conversations and apply them correctly later. | Everyday | Checked automatically | Not in cards | |
| KnowMe-Bench Models read long autobiographical narratives and must infer a person's stable motivations and decision principles, not just retrieve facts. | Everyday | Graded by a model | Not in cards | |
| LongMemEval-V2 Memory systems help agents learn the affordances, dynamics and failure modes of customized web environments. Questions check whether that experience was internalized. | Expert | Graded by a model | Not in cards | |
| MemoryAgentBench Memory agents receive long inputs in chunks over many turns and are tested on accurate retrieval, test-time learning, long-range understanding and selective forgetting. It adds two new sets, EventQA and FactConsolidation. | Everyday | Checked automatically | Not in cards | |
| MemTrapBench Models with memory are tested on cases where correct, relevant memories still distort their reasoning or beliefs and lower task performance. | Everyday | Checked automatically | Not in cards | |
| MINTEval Memory-augmented agents answer questions after long, frequently updated histories where memories interfere. Instances average 138,800 tokens and reach 1.8 million. | Expert | Checked automatically | Not in cards |