Introducing GPT-5 for developers (OpenAI) 7 Aug 2025
- AIME '25 (no tools) 91.1
- FrontierMath (with python tool only) 22.1
- GPQA diamond (no tools) 82.3
- HLE (no tools) 16.7
- HMMT 2025 (no tools) 87.8
- MMMU 81.6
- MMMU-Pro (avg across standard and vision sets) 74.1
- CharXiv reasoning (python enabled) 75.5
- VideoMMMU, max frame 256 82.5
- ERQA 62.9
- SWE-Lancer: IC SWE Diamond Freelance Coding Tasks ($75K) 75000
- SWE-bench Verified (477 of 500 tasks) 71
- Aider polyglot (diff) 71.6
- Scale MultiChallenge (o3-mini grader) 62.3
- Internal API instruction following eval (hard) 65.8
- COLLIE 98.5
- Mean of Tau2-bench airline 60.0 / retail 78.3 / telecom 74.1 70.8
- OpenAI-MRCR: 2 needle 128k 84.3
- Graphwalks bfs <128k 73.4 (parents <128k 64.3) 73.4
- BrowseComp Long Context 128k 89.4 (256k 86.0) 89.4
- VideoMME (long, with subtitle category) 78.5
- Best of LongFact-Concepts / LongFact-Objects hallucination rate, no tools (lower is better) Concepts 0.7, Objects 1.3 0.7
- FActScore hallucination rate, no tools (lower is better) 3.5