- MMLU Pro 81.9
- AA-Omniscience (metric not stated) 17.5
- GPQA Diamond (no tools) 75.4
- HLE (text-only, no tools) 11.7
- SciCode 32.6
- SWE-bench Verified 51.6
- SWE-bench Multilingual 39.3
- Terminal-Bench 2.1 24.6
- PinchBench 85.4
- BrowseComp 37
- τ³-bench (Banking) 9.28
- GDPval-AA-V2 832
- IFBench (loose) 71.9
- AA-LCR 52
Nemotron 3.5 Lightning 30B-A3B
Nemotron 3.5 Lightning 30B-A3B.
49
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Nemotron 3.5 Lightning 30B-A3B
Open weights Small
No Epoch Capabilities Index score yet
- IFBench 72% #36 /70
- SciCode 32.7% #41 /47
- AA-LCR (Artificial Analysis Long Context Reasoning) 52.4% #26 /31
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU-Pro | 81.9% | #56 of 151 | Model card | |
| RULER | 91.5% | #16 of 38 | Model card | |
| IFBench | 72% | #36 of 70 | Model card | |
| LMArena (Elo) | 1,353 | #173 of 333 | Official leaderboard | |
| Creative Writing v3 | 58.0 | #89 of 120 | Official leaderboard | |
| AA-Omniscience | 17.5% | #14 of 17 | Model card | |
| AA-LCR (Artificial Analysis Long Context Reasoning) | 52.4% | #26 of 31 | Model card | |
| Humanity's Last Exam | 11.7% | #127 of 147 | Model card | |
| SEA-HELM | 46.1 | #58 of 62 | Official leaderboard | |
| NV Chat Benchmark | 12.1 | #2 of 2 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Diamond | 76.5% | #130 of 284 | Model card | |
| DTBench | 50.7% | #111 of 178 | Official leaderboard | |
| ProofBench v1.1 | 0% | #51 of 51 | Official leaderboard | |
| LatchBio Capabilities v1.0 | 14.4% | #27 of 27 | Official leaderboard | |
| CritPt | 0% | #14 of 14 | Model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SWE-bench Verified | 52.2% | #123 of 150 | Model card | |
| SciCode | 32.7% | #41 of 47 | Model card | |
| Code Migration | 2.95% | #75 of 78 | Official leaderboard | |
| SWE-bench Multilingual | 39.3% | #69 of 70 | Model card | |
| Vibe Code Bench | 0% | #115 of 115 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PinchBench 2.0 | 86.4% | #11 of 57 | Official leaderboard | |
| PinchBench | 85.4% | #9 of 13 | Model card | |
| BrowseComp | 39.5% | #68 of 77 | Model card | |
| τ³-bench Banking | 9.90% | #18 of 20 | Model card | |
| Terminal-Bench 2.1 | 24.6% | #52 of 53 | Model card |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CorpFin v2 | 54.2% | #87 of 119 | Official leaderboard | |
| Excel Modeling Benchmark | 17% | #72 of 76 | Official leaderboard | |
| Legal Research Bench | 2.88% | #85 of 88 | Official leaderboard | |
| Harvey LAB (Legal Agent Benchmark) | 0% | #78 of 80 | Official leaderboard | |
| Vals Finance Agent v2 | 18.5% | #78 of 80 | Official leaderboard | |
| Tax Agent Bench | 2.59% | #78 of 79 | Official leaderboard | |
| MedScribe | 4.27% | #98 of 98 | Official leaderboard | |
| GDPval-AA v2 | 832 | #26 of 26 | Model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DistressBench | 63.5% | #37 of 42 | Official leaderboard | |
| CyberBench v1.1 (Vals) | 29.5% | #42 of 43 | Official leaderboard | |
| OncologyBench | 0% | #30 of 30 | Official leaderboard | |
| TxBench-Antibody-Discovery | 16% | #28 of 28 | Official leaderboard | |
| TxBench-Oligo-Discovery | 8.55% | #28 of 28 | Official leaderboard | |
| TxBench-Preclinical-Pharmacology | 10.3% | #28 of 28 | Official leaderboard | |
| BioSecBench-Function | 13.8% | #27 of 27 | Official leaderboard | |
| BioSecBench-Surveillance | 14.1% | #27 of 27 | Official leaderboard | |
| EpiBench | 6.92% | #27 of 27 | Official leaderboard | |
| scBench-Long | 0% | #27 of 27 | Official leaderboard | |
| SpatialBench-Long | 4.76% | #27 of 27 | Official leaderboard | |
| VariantBench | 14.4% | #27 of 27 | Official leaderboard | |
| MetagenomicsBench | 7.67% | #26 of 26 | Official leaderboard | |
| BioSecBench-Refusal | 0% | #22 of 22 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ACCoRD | 54.7 | #106 of 150 | Official leaderboard | |
| Conceptual Reasoning Index | 33.6 | #108 of 149 | Official leaderboard |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 4 of them aren't in the launch post.
Capabilities
- SWE-bench Verified 52.2
- SWE-bench Multilingual 37.1
- tau2-bench
- τ³-bench Banking 9.9
- Terminal-Bench 2.1 24.4
- GDPval-AA v2
- PinchBench 83.9
- BrowseComp 39.5
- SciCode 32.7
- GPQA Diamond 76.5
- Humanity's Last Exam 10.8
- CritPt 0
- AA-LCR (Artificial Analysis Long Context Reasoning) 52.4
- RULER 91.5
- IFBench 72
- NV Chat Benchmark 12.1