PaLM (540B)
PaLM (540B).
Google DeepMind Released 4 Apr 2022 13
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Google DeepMind Apr 2022 PaLM (540B)
Closed weights
No Epoch Capabilities Index score yet
- LAMBADA 77.9% #6 /26
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| WinoGrande | 85.1% | #7 of 76 | Epoch AI | |
| ARC-Challenge | 85.2% | #12 of 75 | Epoch AI | |
| HellaSwag | 83.8% | #15 of 74 | Epoch AI | |
| LAMBADA | 77.9% | #6 of 26 | Epoch AI | |
| OpenBookQA | 68% | #10 of 41 | Epoch AI | |
| BoolQ | 91.9% | #1 of 3 | Official leaderboard | |
| SuperGLUE | 90.4% | #1 of 3 | Official leaderboard | |
| WSC | 95.9% | #1 of 3 | Official leaderboard | |
| PIQA | 82.3% | #21 of 58 | Epoch AI | |
| TriviaQA | 81.4% | #14 of 38 | Epoch AI | |
| MMLU | 69.3% | #59 of 127 | Epoch AI | |
| CB | 96% | #2 of 3 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 56.5% | #32 of 82 | Epoch AI |