CMATH
ERNIE-4.5-300B-A47B 96.7% leads since Jun 2025 Fading 1.7K Chinese elementary-school math word problems. Final-answer accuracy.
Saturated.
In the authors’ words
We present the Chinese Elementary School Math Word Problems (CMATH) dataset, comprising 1.7k elementary school-level math word problems with detailed annotations, source from actual Chinese workbooks and exams. This dataset aims to provide a benchmark tool for assessing the following question: to what grade level of elementary school math do the abilities of popular large language models (LLMs) correspond? We evaluate a variety of popular LLMs, including both commercial and open-source options, and discover that only GPT-4 achieves success (accuracy ≥ 60%) across all six elementary school grades, while other models falter at different grade levels. Furthermore, we assess the robustness of several top-performing LLMs by augmenting the original problems in the CMATH dataset with distracting information. Our findings reveal that GPT-4 is able to maintains robustness, while other model fail. We anticipate that our study will expose limitations in LLMs' arithmetic and reasoning capabilities, and promote their ongoing development and advancement.
- Level
- School Students, from grade school to high school
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Short answer A number, word or short string matched to a key.
ERNIE-4.5-300B-A47B leads with 96.7% among models released by Apr 2026
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
The newest model with a score here came out in Apr 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 1 ERNIE-4.5-300B-A47B 96.7%
- 2
DeepSeek-V4-Pro 90.9% - 3 DeepSeek-V3 90.7%
- 4 DeepSeek-V2 (MoE-236B, May 2024) 81.9%
- 5 Moonlight-16B-A3B 81.1%
- 6 DeepSeek LLM 67B 80.3%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | ERNIE-4.5-300B-A47B | 96.7% | Jun 2025 | Model card | |
| 2 | DeepSeek-V4-Pro | 90.9% | Apr 2026 | Model card | |
| 3 | DeepSeek-V3 | 90.7% | Dec 2024 | Model card | |
| 4 | DeepSeek-V2 (MoE-236B, May 2024) | 81.9% | May 2024 | Model card | |
| 5 | Moonlight-16B-A3B | 81.1% | Feb 2025 | Model card | |
| 6 | DeepSeek LLM 67B | 80.3% | Jan 2024 | Model card |
4 models have held number one since its release in Jun 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
The best score passed 85% in Dec 2024
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Apr 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 24 Apr 2026 | DeepSeek-V4-Pro | 90.9 | CMath (EM) |
| 30 Jun 2025 | ERNIE-4.5-300B-A47B | 96.7 | CMath |
| 16 Mar 2025 | ERNIE 4.5 | CMath | |
| 22 Feb 2025 | Moonlight-16B-A3B | 81.1 | CMath |
| 26 Dec 2024 | DeepSeek-V3 | 90.7 | CMath (EM) |
3 system cards report it, from 2 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 29 Jun 2025 | ERNIE-4.5-300B-A47B technical report | 96.7 | CMath (Table 5) | Capabilities |
| 7 May 2024 | DeepSeek-V2 (MoE-236B, May 2024) technical report | 81.9 | CMath 0-shot, DeepSeek-V2 Chat (RL) (Table 3) | Capabilities |
| 5 Jan 2024 | DeepSeek LLM 67B technical report | 80.3 | CMath, DeepSeek 67B Chat (Table 6). 85.4 with tool-integrated reasoning (Table 16) | Capabilities |
More in Math
- IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
- AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
- HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
- MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
- PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.
Qwen3.7-Max 86.5% - USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%