Benchmark Almanac

Fiction.LiveBench

o3-pro 97.2% leads since Jun 2025 Not in model cards

Models answer 36 questions about 30 Fiction.live stories that need inference about plot and characters. Each story is tested at several context lengths.

Made by Fiction.live Released Feb 2025 Reasoning over long documents Last model added Jan 2026 Scores refreshed 31 Oct 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Leaderboard
57
Models scored in Epoch AI data

2 models share the top score of 97.2% among models released by Jan 2026

Best score per model in Epoch AI’s data. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Jan 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

57 models. Scroll sideways for the rest.

  1. 97.2 o3-pro
  2. 97.2 GPT-5
  3. 94.4 Grok 4
  4. 94.4 Grok 4 Fast
  5. 91.7 Gemini 2.5 Pro
  6. 88.9 o3
  7. 86.1 Kimi K2.5
  8. 83.3 o1
  9. 83.3 Claude 3.7 Sonnet
  10. 83.3 QwQ-32B
  11. 83.3 DeepSeek-V3.2-Exp
  12. 77.8 o4-mini
  13. 77.8 Gemini 2.5 Flash
  14. 75.0 DeepSeek-R1
  15. 75.0 Qwen3-235B-A22B-Thinking
  16. 74.2 Qwen3-32B
  17. 69.4 DeepSeek-R1
  18. 69.4 MiniMax-M1-80k
  19. 69.4 GPT-5 mini
  20. 67.7 Qwen3-235B-A22B
  21. 66.7 Qwen2.5-Max
  22. 66.7 GPT-4o
  23. 66.7 Gemini 2.5 Pro
  24. 66.7 Grok-3 mini
  25. 66.7 Gemini 2.5 Pro
  26. 66.7 Kimi K2
  27. 66.7 Qwen3-Max
  28. 63.9 GPT-4.5
  29. 63.9 GPT-4.1
  30. 62.5 Qwen3-14B
  31. 62.1 Qwen3-8B
  32. 61.1 Gemini 2.0 Flash
  33. 61.1 Claude Opus 4
  34. 61.1 Kimi K2
  35. 58.3 Grok 3
  36. 58.3 chutes/GLM-4.5-FP8
  37. 55.6 deepinfra/Qwen3-Next-80B-A3B-Instruct
  38. 52.9 Qwen3-235B-A22B-Instruct
  39. 52.8 Gemini 2.0 Flash Thinking
  40. 52.8 DeepSeek-V3.1
  41. 50.0 o3-mini
  42. 50.0 DeepSeek-V3
  43. 47.2 Gemini 2.5 Flash
  44. 47.2 Gemini 2.5 Flash-Lite
  45. 46.9 Claude Sonnet 4
  46. 46.2 Llama 4 Maverick
  47. 44.4 GPT-4.1 mini
  48. 44.4 gpt-oss-120b
  49. 44.4 GPT-5 nano
  50. 41.7 Gemini 2.0 Pro
  51. 41.7 chutes/Qwen3-Next-80B-A3B-Instruct
  52. 40.6 Qwen3-30B-A3B
  53. 36.0 Llama 4 Scout
  54. 33.3 Llama 3.3 70B
  55. 33.3 Gemma 3 27B
  56. 25.0 GPT-4.1 nano
  57. 25.0 nvidia-nemotron-nano-9b-v2
  1. 1 o3-pro medium 97.2%
  2. 2 GPT-5 medium 97.2%
  3. 3 Grok 4 94.4%
  4. 4 Grok 4 Fast 94.4%
  5. 5 Gemini 2.5 Pro Jun 2025 91.7%
  6. 6 o3 medium 88.9%
  7. 7 Kimi K2.5 86.1%
  8. 8 o1 medium 83.3%
  9. 9 Claude 3.7 Sonnet 8k thinking 83.3%
  10. 10 QwQ-32B 83.3%
  11. 11 DeepSeek-V3.2-Exp high 83.3%
  12. 12 o4-mini medium 77.8%
  13. 13 Gemini 2.5 Flash May 2025 77.8%
  14. 14 DeepSeek-R1 May 2025 75%
  15. 15 Qwen3-235B-A22B-Thinking Jul 2025 75%
  16. 16 Qwen3-32B 74.2%
  17. 17 DeepSeek-R1 69.4%
  18. 18 MiniMax-M1-80k 69.4%
  19. 19 GPT-5 mini medium 69.4%
  20. 20 Qwen3-235B-A22B 67.7%
  21. 21 Qwen2.5-Max 66.7%
  22. 22 GPT-4o Jan 2025 66.7%
  23. 23 Gemini 2.5 Pro Mar 2025 66.7%
  24. 24 Grok-3 mini medium 66.7%
  25. 25 Gemini 2.5 Pro May 2025 66.7%
  26. 26 Kimi K2 Sep 2025 66.7%
  27. 27 Qwen3-Max 66.7%
  28. 28 GPT-4.5 63.9%
  29. 29 GPT-4.1 63.9%
  30. 30 Qwen3-14B 62.5%
  31. 31 Qwen3-8B 62.1%
  32. 32 Gemini 2.0 Flash Feb 2025 61.1%
  33. 33 Claude Opus 4 61.1%
  34. 34 Kimi K2 Jul 2025 61.1%
  35. 35 Grok 3 58.3%
  36. 36 chutes/GLM-4.5-FP8 58.3%
  37. 37 deepinfra/Qwen3-Next-80B-A3B-Instruct 55.6%
  38. 38 Qwen3-235B-A22B-Instruct Jul 2025 52.9%
  39. 39 Gemini 2.0 Flash Thinking Jan 2025 52.8%
  40. 40 DeepSeek-V3.1 52.8%
  41. 41 o3-mini medium 50%
  42. 42 DeepSeek-V3 Mar 2025 50%
  43. 43 Gemini 2.5 Flash Apr 2025 47.2%
  44. 44 Gemini 2.5 Flash-Lite Jun 2025 47.2%
  45. 45 Claude Sonnet 4 46.9%
  46. 46 Llama 4 Maverick 46.2%
  47. 47 GPT-4.1 mini 44.4%
  48. 48 gpt-oss-120b high 44.4%
  49. 49 GPT-5 nano medium 44.4%
  50. 50 Gemini 2.0 Pro 41.7%
  51. 51 chutes/Qwen3-Next-80B-A3B-Instruct 41.7%
  52. 52 Qwen3-30B-A3B 40.6%
  53. 53 Llama 4 Scout 36%
  54. 54 Llama 3.3 70B 33.3%
  55. 55 Gemma 3 27B 33.3%
  56. 56 GPT-4.1 nano 25%
  57. 57 nvidia-nemotron-nano-9b-v2 25%
#ModelScoreSettingDateSource
1 o3-pro 97.2% medium Jun 2025 Epoch AI
2 GPT-5 97.2% medium Aug 2025 Epoch AI
3 Grok 4 94.4% Jul 2025 Epoch AI
4 Grok 4 Fast 94.4% Sep 2025 Epoch AI
5 Gemini 2.5 Pro Jun 2025 91.7% Jun 2025 Epoch AI
6 o3 88.9% medium Apr 2025 Epoch AI
7 Kimi K2.5 86.1% Jan 2026 Epoch AI
8 o1 83.3% medium Dec 2024 Epoch AI
9 Claude 3.7 Sonnet 83.3% 8k thinking Feb 2025 Epoch AI
10 QwQ-32B 83.3% Mar 2025 Epoch AI
11 DeepSeek-V3.2-Exp 83.3% high Sep 2025 Epoch AI
12 o4-mini 77.8% medium Apr 2025 Epoch AI
13 Gemini 2.5 Flash May 2025 77.8% May 2025 Epoch AI
14 DeepSeek-R1 May 2025 75% May 2025 Epoch AI
15 Qwen3-235B-A22B-Thinking Jul 2025 75% Jul 2025 Epoch AI
16 Qwen3-32B 74.2% Apr 2025 Epoch AI
17 DeepSeek-R1 69.4% Jan 2025 Epoch AI
18 MiniMax-M1-80k 69.4% Jun 2025 Epoch AI
19 GPT-5 mini 69.4% medium Aug 2025 Epoch AI
20 Qwen3-235B-A22B 67.7% Apr 2025 Epoch AI
21 Qwen2.5-Max 66.7% Jan 2025 Epoch AI
22 GPT-4o Jan 2025 66.7% Jan 2025 Epoch AI
23 Gemini 2.5 Pro Mar 2025 66.7% Mar 2025 Epoch AI
24 Grok-3 mini 66.7% medium Apr 2025 Epoch AI
25 Gemini 2.5 Pro May 2025 66.7% May 2025 Epoch AI
26 Kimi K2 Sep 2025 66.7% Sep 2025 Epoch AI
27 Qwen3-Max 66.7% Sep 2025 Epoch AI
28 GPT-4.5 63.9% Feb 2025 Epoch AI
29 GPT-4.1 63.9% Apr 2025 Epoch AI
30 Qwen3-14B 62.5% Apr 2025 Epoch AI
31 Qwen3-8B 62.1% Apr 2025 Epoch AI
32 Gemini 2.0 Flash Feb 2025 61.1% Feb 2025 Epoch AI
33 Claude Opus 4 61.1% May 2025 Epoch AI
34 Kimi K2 Jul 2025 61.1% Jul 2025 Epoch AI
35 Grok 3 58.3% Apr 2025 Epoch AI
36 chutes/GLM-4.5-FP8 58.3% Jul 2025 Epoch AI
37 deepinfra/Qwen3-Next-80B-A3B-Instruct 55.6% Sep 2025 Epoch AI
38 Qwen3-235B-A22B-Instruct Jul 2025 52.9% Sep 2025 Epoch AI
39 Gemini 2.0 Flash Thinking Jan 2025 52.8% Jan 2025 Epoch AI
40 DeepSeek-V3.1 52.8% Aug 2025 Epoch AI
41 o3-mini 50% medium Jan 2025 Epoch AI
42 DeepSeek-V3 Mar 2025 50% Mar 2025 Epoch AI
43 Gemini 2.5 Flash Apr 2025 47.2% Apr 2025 Epoch AI
44 Gemini 2.5 Flash-Lite Jun 2025 47.2% Jun 2025 Epoch AI
45 Claude Sonnet 4 46.9% May 2025 Epoch AI
46 Llama 4 Maverick 46.2% Apr 2025 Epoch AI
47 GPT-4.1 mini 44.4% Apr 2025 Epoch AI
48 gpt-oss-120b 44.4% high Aug 2025 Epoch AI
49 GPT-5 nano 44.4% medium Aug 2025 Epoch AI
50 Gemini 2.0 Pro 41.7% Feb 2025 Epoch AI
51 chutes/Qwen3-Next-80B-A3B-Instruct 41.7% Sep 2025 Epoch AI
52 Qwen3-30B-A3B 40.6% Apr 2025 Epoch AI
53 Llama 4 Scout 36% Apr 2025 Epoch AI
54 Llama 3.3 70B 33.3% Dec 2024 Epoch AI
55 Gemma 3 27B 33.3% Mar 2025 Epoch AI
56 GPT-4.1 nano 25% Apr 2025 Epoch AI
57 nvidia-nemotron-nano-9b-v2 25% Aug 2025 Epoch AI

4 models have held number one since its release in Feb 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind
Epoch AI runs Passed 85% in Apr 2025, by o3
o3-pro 97.2%

The best score passed 85% in Apr 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Run or collected by Epoch AI Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) o3-pro Gemini 2.5 Pro o1 Llama 3.3 70B 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) o3-pro o3

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Long context

All 66 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Beam 1.0 66.9%
  • GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 100%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. K-EXAONE 2.0 94.4%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • AA-LCR 1.1 In use Models answer 100 questions that need reasoning across sets of real documents of about 100K tokens. An LLM equality checker with a fixed system prompt compares each answer with the key. Step 5 Preview 88.3%