Benchmark Almanac

HorizonMath

Tencent Hy4 preview 8.80% leads since Aug 2026 In use

Models attack 113 mostly unsolved problems in eight areas of mathematics and the mathematical sciences. A program checks each proposed closed form, construction or bound.

The paper reports six new solutions found by models. Most models score under 10 percent. Tencent reports pass@4.

In the authors’ words

Can AI make progress on important, unsolved mathematical problems? Large language models are now capable of sophisticated mathematical and scientific reasoning, but whether they can perform novel research is still widely debated and underexplored. We introduce HorizonMath, a benchmark of 113 predominantly unsolved problems spanning eight domains in mathematics and the mathematical sciences, paired with an open-source evaluation framework for automated verification. Our benchmark targets the generator-verifier gap: problems where discovery is hard and requires meaningful mathematical insight, but verification is computationally straightforward. This contrasts with most existing research-level benchmarks, which instead rely on formal proof verification or manual review, both of which are expensive to scale. Because these solutions are unknown, HorizonMath is resistant to data contamination, and most state-of-the-art models score under 10%. Using this framework, we identify six novel solutions to research problems that either resolve previously open questions or improve on the best-known published results, with GPT-5.4 Pro and GPT-5.6 Sol each discovering three of these solutions. Across seven frontier model families, reasoning efficiency and behavior also vary substantially. We release HorizonMath as an open challenge and a growing community resource, where each verified solution is a candidate contribution to the mathematical literature.

Abstract of Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification (arXiv:2603.15617, CC0)
Made by University of Oxford , Harvard University , Princeton University and others (Wang et al.) Released Mar 2026 Research mathematics All tasks public Last in a launch post Aug 2026 Last model added Aug 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
3
Models scored in lab cards
2
Launch posts list it, from 1 lab
1
System card reports it, from 1 lab

Tencent Hy4 preview leads with 8.80% among models released by Aug 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Aug 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Tencent Hy4 preview 8.80%
  2. 2 Tencent Hy3 7.10%
  3. 3 Seed2.1 Pro 2.00%
#ModelScoreSettingDateSource
1 Tencent Hy4 preview 8.80% Aug 2026 Model card
2 Tencent Hy3 7.10% Jul 2026 Model card
3 Seed2.1 Pro 2.00% Jun 2026 Model card

3 models have held number one since its release in Mar 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

ByteDance Seed Tencent
Lab claims Not tested since Aug 2026

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Tencent Hy4 preview Tencent Hy3 Seed2.1 Pro 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Tencent Hy4 preview Seed2.1 Pro

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Tencent
DateLaunch postScoreAs the post writes it
28 Aug 2026 Tencent Hy4 preview 8.8 HorizonMath (pass@4)
2 Jul 2026 Tencent Hy3 7.1 HorizonMath (pass@12)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
23 Jun 2026 Seed2.1 Pro model card 2 HorizonMath (Table 6) Capabilities

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%