o3

o3 ranks first on 9 of the 185 benchmarks it has scores on.

OpenAI Released 16 Apr 2025 Family o3 The lab's model card

189

Benchmarks with a score

Ranked first, out of five or more models

In the top three

OpenAI Apr 2025

o3

Closed weights Flagship

146.9 #76 of 271

Epoch Capabilities Index

Humanity's Last Exam 20.3% #20 /47
SWE-bench Verified 62.3% #27 /32
Toolathlon 17% #43 /50

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
ECLeKTic	43.1%	#1 of 35	Official leaderboard
Arena-Hard v2.0	85.9%	#1 of 23	Official leaderboard
Fiction.LiveBench	88.9%	#6 of 57	Epoch AI
WritingBench	80.5	#5 of 45	Official leaderboard
Lech Mazur Creative Writing	8.39	#5 of 44	Epoch AI
MMLU-Pro	85%	#30 of 212	Official leaderboard
WideSearch	5.10%	#1 of 6	Official leaderboard
SimpleQA	49%	#13 of 59	Model card
Wer wird Millionär? LLM Benchmark	112800.0	#18 of 76	Official leaderboard
MMMLU	88.8%	#16 of 62	Model card
Vectara Hallucination Leaderboard (HHEM)	6.80%	#39 of 148	Official leaderboard
LMArena (Elo)	1,432	#90 of 333	Official leaderboard
SimpleQA Verified	52.3%	#16 of 56	Official leaderboard
Slop Score	31.8	#5 of 17	Official leaderboard
EQ-Bench 3	1,500	#23 of 78	Official leaderboard
Creative Writing v3	73.3	#37 of 120	Official leaderboard
LLM Confabulation Leaderboard	14.4%	#19 of 58	Official leaderboard
MultiNRC	45.5%	#13 of 38	Official leaderboard
FACTS Search	74.8%	#11 of 31	Official leaderboard
FACTS Parametric	57.1%	#16 of 45	Official leaderboard
Humanity's Last Exam	20.3%	#20 of 47	Official leaderboard
PersonQA	59%	#4 of 9	Model card
Judgemark v2.1	75.1	#20 of 42	Official leaderboard
Longform Creative Writing (EQ-Bench)	62.7	#57 of 119	Official leaderboard
FaithJudge	17.8%	#18 of 37	Official leaderboard
FACTS Multimodal	39.9%	#19 of 39	Official leaderboard
CL-bench	17.8%	#13 of 24	Official leaderboard
BullshitBench	23.6%	#62 of 107	Official leaderboard
IndicGenBench	50.0	#8 of 13	Official leaderboard
BullshitBench V2	26%	#76 of 122	Official leaderboard
MultiChallenge	56.6%	#22 of 35	Official leaderboard
SimpleQA Hallucination Rate	51%	#8 of 12	Model card
FACTS Benchmark Suite	39.9%	#27 of 40	Official leaderboard
FACTS Grounding	69.6%	#26 of 34	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
MATH Level 5	97.8%	#4 of 94	Epoch AI
AIME 2024	91.6%	#13 of 87	Model card
HiPhO	64.8%	#6 of 35	Official leaderboard
BRUMO 2025	95.8%	#10 of 42	Official leaderboard
PHYBench	34.8%	#7 of 28	Official leaderboard
SMT 2025	87.7%	#14 of 40	Official leaderboard
DTBench	74.7%	#69 of 178	Official leaderboard
GPQA Diamond	81.8%	#85 of 219	Epoch AI
OTIS Mock AIME 2024-2025	84.4%	#79 of 202	Epoch AI
AIME 2025	88.9%	#57 of 143	Model card
HMMT 2025	77.5%	#24 of 53	Official leaderboard
FrontierMath	18.7%	#32 of 70	Epoch AI
SimpleBench	53.1%	#50 of 100	Official leaderboard
IMO 2025	16.7%	#3 of 6	Official leaderboard
CritPt	1.40%	#89 of 160	Epoch AI
CMIMC 2025	82.5%	#19 of 33	Official leaderboard
CORE-Bench	24.4%	#10 of 17	Official leaderboard
ARC-AGI-2	6.53%	#59 of 95	Official leaderboard
ForecastBench	63.5	#54 of 86	Official leaderboard
ARC-AGI	60.8%	#58 of 92	Official leaderboard
FrontierMath Tier 4	2.08%	#39 of 55	Epoch AI
FrontierMath Tiers 1-3 v2	33.3%	#67 of 89	Epoch AI
ATLAS (OpenCompass)	33.8%	#4 of 5	Official leaderboard

Coding

Benchmark	Score	Rank	Source
AutoCodeBench	51.1%	#2 of 34	Official leaderboard
Aider Polyglot	81.3%	#4 of 54	Epoch AI
LiveCodeBench v5	77.4%	#2 of 21	Official leaderboard
LiveCodeBench v6	75.8%	#2 of 21	Official leaderboard
SWE-bench Multimodal	36%	#1 of 9	Official leaderboard
Codeforces rating	2,706	#8 of 41	Model card
METR Time Horizon	91 min	#7 of 31	Official leaderboard
ArtifactsBench	54%	#7 of 20	Official leaderboard
CodeClash	1,343	#3 of 8	Official leaderboard
SWE-Lancer	54%	#4 of 10	Model card
WeirdML	52.4%	#45 of 112	Official leaderboard
OpenAI PRs	44%	#5 of 12	Model card
METR Time Horizon 1.1	120 min	#11 of 26	Official leaderboard
OpenAI Research Engineer Interview: Coding	97%	#3 of 7	Model card
OpenAI Research Engineer Interview: Multiple Choice	80%	#4 of 9	Model card
ALE-Bench	934	#58 of 121	Epoch AI
GSO-Bench	8.82%	#18 of 30	Official leaderboard
LiveSQLBench	23.7%	#22 of 36	Official leaderboard
MLE-bench	31.6%	#8 of 12	Official leaderboard
SWE-Lancer Diamond (SWE Manager)	41%	#3 of 4	Model card
SWE-bench Verified	62.3%	#27 of 32	Epoch AI
TasteVal	8.28×	#17 of 20	Official leaderboard
PaperBench	18%	#12 of 14	Model card

Agents and tools

Benchmark	Score	Rank	Source
Step Race	5	#2 of 59	Official leaderboard
SnakeBench	36	#7 of 183	Official leaderboard
BFCL V4	63%	#7 of 65	Official leaderboard
Game Arena Word Association	352	#6 of 36	Official leaderboard
LisanBench	3308.7	#33 of 133	Official leaderboard
LLM Chess	778	#32 of 124	Official leaderboard
Game Arena Chess Openings	934	#3 of 11	Official leaderboard
VITA-Bench	26.3%	#5 of 18	Official leaderboard
Game Arena Checkers	989	#12 of 38	Official leaderboard
MCP-Universe	26.4%	#10 of 30	Official leaderboard
VitaBench 2.0	40.3%	#7 of 20	Official leaderboard
Game Arena Chess	923	#14 of 39	Official leaderboard
Elimination Game	4	#21 of 56	Official leaderboard
ITBench	51.7%	#5 of 13	Official leaderboard
MCPMark	25.4%	#11 of 28	Official leaderboard
Mystery Game Puzzles	29%	#32 of 80	Epoch AI
Pencil Puzzle Bench	3.30%	#23 of 53	Official leaderboard
tau-bench	61.2%	#10 of 23	Model card
GAIA	62.1%	#25 of 57	Official leaderboard
Chess Puzzles	37%	#37 of 83	Epoch AI
Game Arena Five in a Row	540	#9 of 20	Official leaderboard
Game Arena Poker	1.17	#14 of 31	Official leaderboard
Game Arena Bargaining	193	#17 of 36	Official leaderboard
OSWorld-Verified	56%	#16 of 33	Official leaderboard
LMArena Search Arena	1,196	#19 of 35	Official leaderboard
Game Arena Reversi	457	#20 of 36	Official leaderboard
Kaggle Game Arena	268	#22 of 39	Official leaderboard
Galileo Agent Leaderboard	0.86	#18 of 31	Official leaderboard
Game Arena Dark Hex	221	#23 of 38	Official leaderboard
Game Arena Four in a Row	294	#23 of 38	Official leaderboard
Game Arena Nine Men's Morris	561	#22 of 36	Official leaderboard
Game Arena Ultimate Tic-Tac-Toe	243	#23 of 36	Official leaderboard
Game Arena Word Art	276	#23 of 35	Official leaderboard
DeepResearch Bench	45.2%	#16 of 24	Epoch AI
Game Arena Go	554	#25 of 36	Official leaderboard
Game Arena Lines of Action	412	#25 of 36	Official leaderboard
BrowseComp	49.7%	#58 of 77	Model card
Game Arena Clobber	141	#28 of 36	Official leaderboard
Toolathlon	17%	#43 of 50	Official leaderboard
Game Arena Dots and Boxes	594	#32 of 37	Official leaderboard
OSWorld	23%	#8 of 9	Epoch AI
Game Arena Coin Game	147	#33 of 35	Official leaderboard

Professional work

Benchmark	Score	Rank	Source
CadEval	74%	#1 of 15	Epoch AI
xbench-Recruitment	78.5	#1 of 8	Official leaderboard
xbench-Marketing	50.8	#1 of 7	Official leaderboard
FailureSensorIQ	67.6%	#4 of 24	Official leaderboard
Vals Finance Agent v1	47.7%	#8 of 47	Official leaderboard
TaxEval v2	74.6%	#34 of 129	Official leaderboard
ProfBench	52.4%	#22 of 72	Official leaderboard
MedCode	47.3%	#33 of 96	Official leaderboard
LegalBench	83.8%	#48 of 138	Official leaderboard
PRBench	48.6%	#16 of 44	Official leaderboard
MortgageTax	65.7%	#37 of 86	Official leaderboard
StockBench	1.90%	#6 of 13	Official leaderboard
CorpFin v2	59.7%	#73 of 119	Official leaderboard
GDPval	30.8%	#7 of 11	Epoch AI
MedScribe	76.7%	#67 of 98	Official leaderboard
Vending-Bench	$1,843	#8 of 9	Official leaderboard
SpreadsheetBench	23.3%	#2 of 2	Official leaderboard

Vision, audio and robotics

Benchmark	Score	Rank	Source
MMMU	82.9%	#4 of 136	Official leaderboard
VHELM	81.0	#2 of 41	Official leaderboard
VisualPuzzles	54%	#2 of 29	Official leaderboard
MathVista	86.8%	#8 of 64	Model card
Video-MMMU	83.3%	#3 of 23	Official leaderboard
MMMU-Pro	76.4%	#10 of 57	Official leaderboard
ERQA	41.5%	#3 of 17	Official leaderboard
Math-VR	59.3%	#5 of 26	Official leaderboard
VISTA	50.1%	#11 of 54	Official leaderboard
VisuLogic	29.5%	#4 of 17	Official leaderboard
EnigmaEval	13.1%	#10 of 41	Epoch AI
VPCT	52%	#7 of 28	Epoch AI
GeoBench	74%	#21 of 48	Official leaderboard
Vision Arena	1,216	#68 of 136	Official leaderboard
ZeroBench	6.00%	#36 of 66	Official leaderboard
CharXiv	78.6%	#37 of 67	Model card
Drone-Bench	55%	#12 of 19	Official leaderboard
Point-Bench	43.5%	#11 of 17	Official leaderboard
ClockBench	12.2%	#25 of 37	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
HELM Safety	98.2	#2 of 75	Official leaderboard
Multimodal Troubleshooting Virology	59%	#2 of 26	Model card
Capture the Flag (Collegiate)	68%	#1 of 13	Model card
Virology Capabilities Test (VCT)	43.8%	#1 of 7	Official leaderboard
Spiral-Bench v1.0	86.1	#2 of 13	Official leaderboard
BaxBench	46.4%	#6 of 34	Official leaderboard
AIR-Bench 2024	84.5%	#15 of 82	Official leaderboard
Spiral-Bench v1.2	63.4	#4 of 21	Official leaderboard
Capture the Flag (High School)	82%	#2 of 9	Model card
FORTRESS	16%	#15 of 66	Official leaderboard
StrongREJECT	97%	#5 of 22	Model card
Human Sourced Jailbreaks	100%	#2 of 8	Model card
PCT (Polarized Contrastive Pairs)	39%	#13 of 46	Official leaderboard
Capture the Flag (Professional)	55%	#9 of 21	Model card
SimpleSafetyTests	99%	#37 of 75	Official leaderboard
Image generation refusals	98%	#1 of 2	Model card
Challenging Refusal Evaluation	92%	#5 of 9	Model card
Tacit Knowledge and Troubleshooting	70%	#17 of 29	Model card
ProtocolQA Open-Ended	30%	#16 of 25	Model card
Apollo Research sabotage evaluations	0.58	#2 of 3	Model card
Anthropic Red Team	98.3%	#52 of 75	Official leaderboard
Cyber Range	0%	#8 of 10	Model card
First-Person Fairness evaluation	0.60%	#9 of 10	Model card

Other

Benchmark	Score	Rank	Source
CAIS Risk Index	30.2	#2 of 65	Official leaderboard
HELM Capabilities	81.1	#3 of 60	Official leaderboard
ACCoRD	71.4	#41 of 150	Official leaderboard
LMCA	39.7%	#52 of 139	Epoch AI
Conceptual Reasoning Index	53.0	#57 of 149	Official leaderboard
CAIS Text Capabilities Index	25.6	#42 of 66	Official leaderboard
CAIS Vision Capabilities Index	46.5	#35 of 53	Official leaderboard
SAGE (Vals)	41.8%	#56 of 81	Official leaderboard

OpenAI's models over time

OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Introducing OpenAI o3 and o4-mini 16 Apr 2025

Its system card

Every result in the longer documents the lab published with it. 33 of them aren't in the launch post.

o3

o3

Where it ranks

Knowledge and language

Math and science

Coding

Agents and tools

Professional work

Vision, audio and robotics

Safety and security

Other

OpenAI's models over time

Its launch post

Its system card

Capabilities

Hallucination

Safety

Robustness

Bias

Alignment

Bio and chem risk

Cyber risk

AI R&D risk

Autonomy