Grok-2 Dec 2024
Grok-2.
xAI Released 12 Dec 2024
23
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
xAI Dec 2024
Grok-2
Dec 2024 snapshot Closed weights
130.6
#159 of 271
Epoch Capabilities Index
- SimpleQA Verified 20.5% #40 /56
- GPQA Diamond 53.8% #140 /219
- FrontierMath 0.69% #62 /70
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU-Pro | 75.5% | #73 of 212 | Official leaderboard | |
| BuzzBench | 51.8 | #10 of 23 | Official leaderboard | |
| ECLeKTic | 18.4% | #18 of 35 | Official leaderboard | |
| FACTS Grounding | 76.2% | #19 of 34 | Official leaderboard | |
| LLM Confabulation Leaderboard | 20.1% | #38 of 58 | Official leaderboard | |
| SimpleQA Verified | 20.5% | #40 of 56 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 1.90% | #128 of 148 | Official leaderboard | |
| Lech Mazur Creative Writing | 6.36 | #40 of 44 | Epoch AI | |
| Phare | 54.6% | #61 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH Level 5 | 63.5% | #40 of 94 | Epoch AI | |
| GPQA Diamond | 53.8% | #140 of 219 | Epoch AI | |
| DTBench | 42% | #122 of 178 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 11.5% | #151 of 202 | Epoch AI | |
| FrontierMath | 0.69% | #62 of 70 | Epoch AI | |
| SimpleBench | 22.7% | #74 of 82 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| WeirdML | 22.2% | #113 of 130 | Epoch AI |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Elimination Game | 3 | #42 of 56 | Official leaderboard | |
| LLM Chess | -335 | #106 of 124 | Official leaderboard | |
| Step Race | 1 | #56 of 59 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TaxEval v2 | 67% | #100 of 129 | Official leaderboard | |
| CorpFin v2 | 51.1% | #93 of 119 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BaxBench | 18.4% | #23 of 34 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 54.3% | #20 of 48 | Epoch AI |
xAI's models over time
xAI's models in color, against every other model on the Epoch Capabilities Index, by release date.