Benchmark Almanac

FrontierFinance

Samaya Sterling 58.9% tops its leaderboard Claude Fable 5.1 55.9% leads the models we track In use

Agents answer 220 investment research queries from finance experts across six use cases of the investor workflow. Three judge models check each answer against 11,543 source-attributed rubrics.

The score is the rubric qualification rate, macro-averaged over queries. Results are reported per harness: web search only, the open Finance Agent v2 harness, and Samaya in-house harness.

In the authors’ words

AI agents are increasingly deployed for professional investment research, yet no benchmark captures the complexity of the full investor workflow. Existing benchmarks mainly target financial data extraction, a narrow slice that current models have largely saturated, while reference-based metrics and generic LLM-as-a-judge scoring fall short on the open-ended, long-form answers that real analyst queries demand. We introduce FrontierFinance, a fully open benchmark of 220 expert-crafted queries and 11,543 source-attributed rubrics spanning six crucial use cases across the full investor workflow. FrontierFinance is both broader and harder than existing public finance benchmarks. Evaluating frontier models and agent systems under a common harness restricted to publicly available data, we find that the tool harness, not the model alone, strongly shapes quality and efficiency; that Samaya's in-house system leads at 56.0%, ahead of the strongest frontier model (Claude Fable 5, 49.2%) at roughly 2.2x lower cost; and that the best open-weight model (Kimi K3, 46.4%) nearly matches the best proprietary model at 4.5x lower cost. Screening & Discovery and Sector, Industry & Macro remain the hardest use cases across all systems, where even the best systems reach only 33% and 39%. We make the dataset and grading code publicly available.

Abstract of FrontierFinance: A Challenging Benchmark for Measuring Frontier Intelligence of Finance Agents (arXiv:2608.11683, CC0)
Made by Samaya AI Released Jul 2026 Finance All tasks public Last in a launch post Oct 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
19
Entries on its own leaderboard, 16 models we track
2
Launch posts list it, from 2 labs

Samaya Sterling tops the leaderboard with 58.9%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Claude Fable 5.1, at 55.9%.

  1. 1 Samaya Sterling 58.9%
  2. 2 Legacy Samaya System (high effort) 56%
  3. 3 Claude Fable 5.1 high 55.9%
  4. 4 Muse Spark Open xhigh 53.7%
  5. 5 Legacy Samaya System 52.9%
  6. 6 Muse Wonder 52.6%
  7. 7 Claude Opus 5.5 high 50.9%
  8. 8 Gemini 3.8 Flash medium 50%
  9. 9 Claude Fable 5 high 49.2%
  10. 10 GPT-5.6 Sol medium 46.8%
  11. 11 Ling-3.0-flash-Fin 46.6%
  12. 12 Kimi K3 max 46.4%
  13. 13 Gemini 3.6 Flash medium 46.3%
  14. 14 Claude Opus 4.8 high 45%
  15. 15 GPT-5.5 medium 43.5%
  16. 16 GLM-5.2 max 42.8%
  17. 17 DeepSeek-V4-Pro high 40.5%
  18. 18 Kimi K2.6 on; level not supported 32.3%
  19. 19 Gemini 3.1 Pro high 30.7%
#EntryScoreSettingDateSource
1 Samaya Sterling 58.9% Official leaderboard
2 Legacy Samaya System (high effort) 56% Official leaderboard
3 Claude Fable 5.1 55.9% high, Finance Agent v2 Sep 2026 Official leaderboard
4 Muse Spark Open 53.7% xhigh, Finance Agent v2 Oct 2026 Official leaderboard
5 Legacy Samaya System 52.9% Official leaderboard
6 Muse Wonder 52.6% Finance Agent v2 Oct 2026 Official leaderboard
7 Claude Opus 5.5 50.9% high, Finance Agent v2 Sep 2026 Official leaderboard
8 Gemini 3.8 Flash 50% medium, Finance Agent v2 Sep 2026 Official leaderboard
9 Claude Fable 5 49.2% high, Finance Agent v2 Jun 2026 Official leaderboard
10 GPT-5.6 Sol 46.8% medium, Finance Agent v2 Jul 2026 Official leaderboard
11 Ling-3.0-flash-Fin 46.6% Finance Agent v2 Sep 2026 Official leaderboard
12 Kimi K3 46.4% max, Finance Agent v2 Jul 2026 Official leaderboard
13 Gemini 3.6 Flash 46.3% medium, Finance Agent v2 Jul 2026 Official leaderboard
14 Claude Opus 4.8 45% high, Finance Agent v2 May 2026 Official leaderboard
15 GPT-5.5 43.5% medium, Finance Agent v2 Apr 2026 Official leaderboard
16 GLM-5.2 42.8% max, Finance Agent v2 Jun 2026 Official leaderboard
17 DeepSeek-V4-Pro 40.5% high, Finance Agent v2 Apr 2026 Official leaderboard
18 Kimi K2.6 32.3% on; level not supported, Finance Agent v2 Apr 2026 Official leaderboard
19 Gemini 3.1 Pro 30.7% high, Web Search Feb 2026 Official leaderboard

Labs report scores for 2 models in their cards. We plot every source in the charts below.

The best score, 55.9%, costs $6.89 a task

Runs on the cost frontier score higher than any run that's cheaper. Muse Spark Open at xhigh comes within 2.2 points of the best for $1.38 a task, 5 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic Google DeepMind Ant Group Meta Cost frontier One model at each effort level
0% 20% 40% 60% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.1, high Muse Spark Open, xhigh Gemini 3.8 Flash, medium Ling-3.0-flash-Fin 0% 20% 40% 60% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.1, high Gemini 3.8 Flash, medium
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.1 high Finance Agent v2 55.9% $6.89
Muse Spark Open xhigh Finance Agent v2 53.7% $1.38
Muse Wonder Finance Agent v2 52.6% $3.35
Claude Opus 5.5 high Finance Agent v2 50.9% $1.87
Gemini 3.8 Flash medium Finance Agent v2 50% $1.21
Claude Fable 5 high Finance Agent v2 49.2% $4.06
GPT-5.6 Sol medium Finance Agent v2 46.8% $3.03
Ling-3.0-flash-Fin Finance Agent v2 46.6% $0.05
Kimi K3 max Finance Agent v2 46.4% $0.90
Gemini 3.6 Flash medium Finance Agent v2 46.3% $2.41
Claude Opus 4.8 high Finance Agent v2 45% $2.61
GPT-5.5 medium Finance Agent v2 43.5% $2.80
GLM-5.2 max Finance Agent v2 42.8% $0.63
DeepSeek-V4-Pro high Finance Agent v2 40.5% $0.68
Claude Opus 4.8 high Web Search 33% $1.47
Kimi K2.6 on; level not supported Finance Agent v2 32.3% $0.95
Gemini 3.1 Pro high Web Search 30.7% $0.24
Gemini 3.1 Pro high Finance Agent v2 30.5% $1.72
GPT-5.5 medium Web Search 20.7% $0.55

2 models have held number one since its release in Jul 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 2 leaders

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Step 5 Preview Claude Fable 5.1 Claude Fable 5 Claude Opus 4.8 Kimi K2.6 Gemini 3.1 Pro 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Step 5 Preview Claude Fable 5.1

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

StepFun
Moonshot AI
DateLaunch postScoreAs the post writes it
24 Oct 2026 Kimi K3.1 Pro 58.6 Frontier Finance
20 Sep 2026 Step 5 Preview 66.4 FrontierFinance

More in Law, finance and medicine

All 140 in Law, finance and medicine
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
  • PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Muse Spark 1.3 61.6%