Announcing Grok-1.5 28 Mar 2024
- MMLU (5-shot) 81.3
- MATH (4-shot, maj@1) 50.6
- GSM8K (8-shot, maj@1) 90
- HumanEval (0-shot, pass@1) 74.1
Grok-1.5
Grok-1.5.
4
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
xAI Mar 2024
Grok-1.5
Flagship
No Epoch Capabilities Index score yet
- MMLU 81.3% #67 /127
- HumanEval 74.1% #59 /95
- MATH 50.6% #61 /93
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 81.3% | #67 of 127 | Model card |
Math and science
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 74.1% | #59 of 95 | Model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.