PaLM 2-L

PaLM 2-L ranks first on 2 of the 17 benchmarks it has scores on.

Released 17 May 2023 Family PaLM 2 The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Unknown lab May 2023

Flagship

114.9 #216 of 271

Epoch Capabilities Index

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
BoolQ	90.9%	#1 of 28	Model card
WinoGrande	90.9%	#2 of 42	Model card
CommonSenseQA	90.4%	#1 of 17	Model card
ARC-Challenge	95.1%	#5 of 45	Model card
PIQA	85%	#6 of 33	Model card
TriviaQA	86.1%	#8 of 39	Model card
ANLI	73.1%	#1 of 4	Model card
ARC-Easy	89.7%	#6 of 23	Model card
Natural Questions	37.5%	#6 of 20	Model card
RACE-M	77%	#1 of 3	Model card
HellaSwag	86.8%	#17 of 47	Model card
DROP	85%	#15 of 40	Model card
MGSM	87%	#20 of 52	Model card
SQuAD v2	80.5%	#4 of 8	Model card
StrategyQA	90.4%	#1 of 2	Model card
XCOPA	94.4%	#1 of 2	Model card
BIG-Bench Hard	78.1%	#38 of 71	Model card
OpenBookQA	58.5%	#12 of 20	Model card
RACE-H	62.3%	#5 of 7	Model card
CB	87.5%	#1 of 1	Model card
COPA	96%	#1 of 1	Model card
LAMBADA	86.9%	#1 of 1	Model card
MultiRC	88.2%	#1 of 1	Model card
ReCoRD	93.8%	#1 of 1	Model card
RTE	79.3%	#1 of 1	Model card
StoryCloze	87.4%	#1 of 1	Model card
WebQuestions	28.2%	#1 of 1	Model card
WiC	66.8%	#1 of 1	Model card
Winograd	89.5%	#1 of 1	Model card
WSC	86.9%	#1 of 1	Model card

Benchmark	Score		Rank	Source
GSM8K	91%		#35 of 87	Model card
MATH	48.8%		#65 of 93	Model card

's models in color, against every other model on the Epoch Capabilities Index, by release date.

The benchmarks the lab chose to headline, with the scores it gave.