Benchmark Almanac

AA-LCR (Artificial Analysis Long Context Reasoning)

Muse Glimmer 80% leads since Aug 2026 In use

Questions requiring reasoning across multiple real documents totaling about 100K tokens. Answers judged by an LLM.

Version 1.0.0 stays available at dataset revision bdae010. Version 1.1 (September 2026) fixed 16 answer keys, so scores differ.

Made by Artificial Analysis Released Aug 2025 Reasoning over long documents All tasks public Last in a launch post Oct 2026 Last model added Oct 2026
Level
University Undergraduates and entry-level professionals
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Dataset Leaderboard
31
Models scored in lab cards
29
Launch posts list it, from 13 labs
18
System cards report it, from 7 labs

Muse Glimmer leads with 80%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

31 models. Scroll sideways for the rest.

  1. 80.0 Muse Glimmer
  2. 79.3 Beam 1.0
  3. 74.7 Kimi K3
  4. 73.4 Tencent Hy3
  5. 72.0 MiniMax-M2.7
  6. 71.2 Mistral Small 4
  7. 71.0 Solar Pro 4
  8. 69.5 MiniMax-M2.5
  9. 68.7 Qwen3-Max
  10. 68.7 Qwen3.5 397B-A17B
  11. 68.3 Qwen 3.6 Plus
  12. 68.0 Mercury 2.5
  13. 66.9 Qwen3.5-122B-A10B
  14. 66.3 Tencent Hy3 preview
  15. 65.4 Nemotron 3 Ultra
  16. 64.1 Nemotron 3.5 Bolt 74B-A9B
  17. 63.9 Step 3.7 Flash
  18. 62.3 Solar Open 2
  19. 62.0 MiniMax-M2.1
  20. 61.0 MiniMax-M2
  21. 58.3 Nemotron 3 Super
  22. 57.0 Qwen3.5-Omni-Plus
  23. 56.2 K-EXAONE 2.0
  24. 53.5 K-EXAONE
  25. 52.8 LFM3-1.7B
  26. 52.4 Nemotron 3.5 Lightning 30B-A3B
  27. 50.6 EXAONE 4.5 33B
  28. 41.0 Nemotron 3 Nano Omni 30B-A3B
  29. 39.1 Nemotron-Cascade 2 30B-A3B
  30. 35.9 Nemotron 3 Nano 30B-A3B
  31. 35.0 Solar Open 100B
  1. 1 Muse Glimmer 80%
  2. 2 Beam 1.0 79.3%
  3. 3 Kimi K3 74.7%
  4. 4 Tencent Hy3 73.4%
  5. 5 MiniMax-M2.7 72%
  6. 6 Mistral Small 4 71.2%
  7. 7 Solar Pro 4 71%
  8. 8 MiniMax-M2.5 69.5%
  9. 9 Qwen3-Max 68.7%
  10. 10 Qwen3.5 397B-A17B 68.7%
  11. 11 Qwen 3.6 Plus 68.3%
  12. 12 Mercury 2.5 68%
  13. 13 Qwen3.5-122B-A10B 66.9%
  14. 14 Tencent Hy3 preview 66.3%
  15. 15 Nemotron 3 Ultra 65.4%
  16. 16 Nemotron 3.5 Bolt 74B-A9B 64.1%
  17. 17 Step 3.7 Flash 63.9%
  18. 18 Solar Open 2 62.3%
  19. 19 MiniMax-M2.1 62%
  20. 20 MiniMax-M2 61%
  21. 21 Nemotron 3 Super 58.3%
  22. 22 Qwen3.5-Omni-Plus 57%
  23. 23 K-EXAONE 2.0 56.2%
  24. 24 K-EXAONE 53.5%
  25. 25 LFM3-1.7B 52.8%
  26. 26 Nemotron 3.5 Lightning 30B-A3B 52.4%
  27. 27 EXAONE 4.5 33B 50.6%
  28. 28 Nemotron 3 Nano Omni 30B-A3B 41%
  29. 29 Nemotron-Cascade 2 30B-A3B 39.1%
  30. 30 Nemotron 3 Nano 30B-A3B 35.9%
  31. 31 Solar Open 100B 35%
#ModelScoreSettingDateSource
1 Muse Glimmer 80% Aug 2026 Model card
2 Beam 1.0 79.3% Oct 2026 Model card
3 Kimi K3 74.7% Jul 2026 Model card
4 Tencent Hy3 73.4% Jul 2026 Model card
5 MiniMax-M2.7 72% May 2026 Model card
6 Mistral Small 4 71.2% Mar 2026 Model card
7 Solar Pro 4 71% Aug 2026 Model card
8 MiniMax-M2.5 69.5% Feb 2026 Model card
9 Qwen3-Max 68.7% Jan 2026 Model card
10 Qwen3.5 397B-A17B 68.7% Feb 2026 Model card
11 Qwen 3.6 Plus 68.3% Apr 2026 Model card
12 Mercury 2.5 68% Sep 2026 Model card
13 Qwen3.5-122B-A10B 66.9% Feb 2026 Model card
14 Tencent Hy3 preview 66.3% Apr 2026 Model card
15 Nemotron 3 Ultra 65.4% Jun 2026 Model card
16 Nemotron 3.5 Bolt 74B-A9B 64.1% Oct 2026 Model card
17 Step 3.7 Flash 63.9% May 2026 Model card
18 Solar Open 2 62.3% Jul 2026 Model card
19 MiniMax-M2.1 62% Dec 2025 Model card
20 MiniMax-M2 61% Oct 2025 Model card
21 Nemotron 3 Super 58.3% Mar 2026 Model card
22 Qwen3.5-Omni-Plus 57% Mar 2026 Model card
23 K-EXAONE 2.0 56.2% Jul 2026 Model card
24 K-EXAONE 53.5% Dec 2025 Model card
25 LFM3-1.7B 52.8% Oct 2026 Model card
26 Nemotron 3.5 Lightning 30B-A3B 52.4% Oct 2026 Model card
27 EXAONE 4.5 33B 50.6% Apr 2026 Model card
28 Nemotron 3 Nano Omni 30B-A3B 41% Apr 2026 Model card
29 Nemotron-Cascade 2 30B-A3B 39.1% Mar 2026 Model card
30 Nemotron 3 Nano 30B-A3B 35.9% Dec 2025 Model card
31 Solar Open 100B 35% Dec 2025 Model card

9 models have held number one since its release in Aug 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

MiniMax Qwen Mistral AI Tencent Moonshot AI Meta
Lab claims 9 leaders
Muse Glimmer 80%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Muse Glimmer Kimi K3 Mistral Small 4 MiniMax-M2.1 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Muse Glimmer Tencent Hy3

9 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

MiniMax
NVIDIA
LG AI Research
Upstage
Qwen
Mistral AI
Tencent
StepFun
Moonshot AI
Meta
Inception Labs
Reflection
Liquid AI
DateLaunch postScoreAs the post writes it
27 Oct 2026 LFM3-1.7B 52.8 AA-LCR
15 Oct 2026 Nemotron 3.5 Bolt 74B-A9B 64.1 AA-LCR
5 Oct 2026 Beam 1.0 79.3 AA-LCR
8 Sep 2026 Mercury 2.5 68 AA-LCR (version not stated)
11 Aug 2026 Nemotron 3.5 Lightning 30B-A3B 52 AA-LCR
11 Aug 2026 Solar Pro 4 71 AA-LCR (Artificial Analysis score quoted by Upstage)
10 Aug 2026 Muse Glimmer 80 AA-LCR
31 Jul 2026 K-EXAONE 2.0 56.2 AA-LCR
22 Jul 2026 Solar Open 2 62.3 AA-LCR
16 Jul 2026 Kimi K3 74.7 AA-LCR
2 Jul 2026 Tencent Hy3 73.4 AA-LCR
4 Jun 2026 Nemotron 3 Ultra 65.4 AA-LCR
29 May 2026 Step 3.7 Flash 63.9 AA-LCR (avg@16/acc)
23 Apr 2026 Tencent Hy3 preview 66.3 AA-LCR
9 Apr 2026 EXAONE 4.5 33B 50.6 AA-LCR
2 Apr 2026 Qwen 3.6 Plus 68.3 AA-LCR
30 Mar 2026 Qwen3.5-Omni-Plus 57 AA-LCR
19 Mar 2026 Nemotron-Cascade 2 30B-A3B 39.1 AA-LCR
16 Mar 2026 Mistral Small 4 71.2 LCR [Mistral Small 4 - High] (text: 'AA LCR ... 0.72')
11 Mar 2026 Nemotron 3 Super 58.3 AA-LCR
24 Feb 2026 Qwen3.5-122B-A10B 66.9 AA-LCR
16 Feb 2026 Qwen3.5 397B-A17B 68.7 AA-LCR
12 Feb 2026 MiniMax-M2.5 69.5 AA-LCR
23 Jan 2026 Qwen3-Max 68.7 AA-LCR
31 Dec 2025 Solar Open 100B 35 AA-LCR
31 Dec 2025 K-EXAONE 53.5 AA-LCR
23 Dec 2025 MiniMax-M2.1 62 AA-LCR
15 Dec 2025 Nemotron 3 Nano 30B-A3B 35.9 AA-LCR
27 Oct 2025 MiniMax-M2 61 AA-LCR

18 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
19 Oct 2026 Muse Glimmer technical report 80 AA-LCR, high (Tables 2 and 5) Capabilities
15 Oct 2026 Nemotron 3.5 Lightning 30B-A3B technical report 52.4 AA-LCR, BF16 acc. (Table 22) Capabilities
15 Oct 2026 Nemotron 3.5 Bolt 74B-A9B technical report 64.1 AA-LCR, released 24GB checkpoint (Table 7). BF16 67.38 Capabilities
5 Aug 2026 K-EXAONE 2.0 technical report 56.2 AA-LCR (Table 6, Reasoning mode) Capabilities
27 Jul 2026 Kimi K3 technical report 74.7 AA-LCR, cited from Artificial Analysis (Table 2) Capabilities
22 Jul 2026 Solar Open 2 technical report 62.3 AA-LCR (Table 3) Capabilities
9 Jun 2026 Nemotron 3 Ultra technical report 65.4 AA-LCR (Table 10) Capabilities
26 May 2026 MiniMax-M2 technical report 61 AA-LCR, MiniMax-M2 bar, printed label (Figure 9) Capabilities
26 May 2026 MiniMax-M2.7 technical report 72 AA-LCR, AA Index v4.0 protocol (Table 4) Capabilities
26 May 2026 MiniMax-M2.5 technical report 65 AA-LCR, AA Index v4.0 protocol (Table 4) Capabilities
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report 41 AA-LCR (Table 10) Capabilities
17 Apr 2026 Qwen3.5-Omni-Plus technical report 57 AA-LCR (Table 4) Capabilities
9 Apr 2026 EXAONE 4.5 33B technical report 50.6 AA-LCR (Table 3, Reasoning mode) Capabilities
19 Mar 2026 Nemotron-Cascade 2 30B-A3B technical report 39.1 AA-LCR (Table 1) Capabilities
11 Mar 2026 Nemotron 3 Super technical report 58.3 AA-LCR (Table 5) Capabilities
11 Jan 2026 Solar Open 100B technical report 35 AA-LCR (Table 6) Capabilities
5 Jan 2026 K-EXAONE technical report 53.5 AA-LCR (Table 3, Reasoning mode). Non-reasoning 45.2 (Table 4) Capabilities
23 Dec 2025 Nemotron 3 Nano 30B-A3B technical report 35.9 AA-LCR, reasoning on (Table 3) Capabilities

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

MiniMax Qwen Mistral AI Tencent Moonshot AI Meta StepFun
AA-LCR 1.1 Passed 85% in Sep 2026, by Step 5 Preview
Step 5 Preview 88.3%
AA-LCR (Artificial Analysis Long Context Reasoning) 9 leaders
Muse Glimmer 80%
AA-LCR 1.1 Passed 85% in Sep 2026, by Step 5 Preview
Step 5 Preview 88.3%
Replaced by AA-LCR 1.1

More in Long context

All 66 in Long context
  • MRCR v2 In use Multi-round coreference: retrieve the n-th of several similar requests hidden in a long synthetic conversation, up to 1M tokens. Muse Spark 1.3 98.5%
  • LongBench v2 In use 503 multiple-choice questions over 8K-2M-word contexts (QA, dialogue, code repos, structured data). Accuracy. Beam 1.0 66.9%
  • GraphWalks In use A graph is given as an edge list in long context. The model answers BFS or parent queries, scored by F1. Claude Fable 5 100%
  • OpenAI MRCR v2 In use Models read a long synthetic chat that hides 2, 4 or 8 identical writing requests and must reproduce the response to the i-th one. Scored by mean string match ratio per context bin. K-EXAONE 2.0 94.4%
  • CL-bench In use Model must learn new knowledge from a supplied long context (rules, procedures, data) and solve tasks. Strict LLM-verified rubrics. GPT-5.4 27.9%
  • LOCA-Bench In use Agentic tasks whose environment state is scaled to grow context length while task semantics stay fixed. Success rate versus context length. GPT-5.2 21.3%