Gemini 2.0 Flash places in the top three on 3 benchmarks.
Google DeepMind Dec 2024 Dec 2024 snapshot Closed weights Mid-size
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SEAL Multilingual | 1,119 | #2 of 19 | Official leaderboard | |
| MMLU | 79.7% | #25 of 127 | Epoch AI | |
| SEAL Precise Instruction Following | 86.6 | #4 of 18 | Official leaderboard | |
| LongBench v2 | 51.1% | #12 of 33 | Official leaderboard | |
| MMBench-CN v1.1 | 71.6% | #92 of 178 | Official leaderboard | |
| FACTS Grounding | 83.6% | #8 of 15 | Model card | |
| MRCR (Michelangelo, original) | 69.2% | #5 of 9 | Model card | |
| MMLU-Pro | 76.4% | #96 of 151 | Model card | |
| Lech Mazur Creative Writing | 7.15 | #35 of 44 | Epoch AI | |
| Vectara Hallucination Leaderboard (HHEM) | 1.30% | #140 of 148 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 89.7% | #9 of 93 | Model card | |
| HiddenMath | 63% | #4 of 11 | Model card | |
| GPQA Diamond | 62.1% | #202 of 284 | Model card | |
| ARC-AGI-2 | 1.30% | #76 of 95 | Official leaderboard | |
| SimpleBench | 18.9% | #94 of 100 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 52.9% | #8 of 69 | Official leaderboard | |
| SEAL Coding | 1,111 | #5 of 26 | Official leaderboard | |
| Natural2Code | 92.9% | #2 of 9 | Model card | |
| BigCodeBench-Hard | 28.4% | #26 of 96 | Official leaderboard | |
| Aider-Edit | 69.9% | #12 of 37 | Official leaderboard | |
| Bird-SQL | 56.9% | #7 of 13 | Model card | |
| SWE-bench Verified | 52.2% | #35 of 60 | Official leaderboard | |
| Aider Polyglot | 38.2% | #34 of 54 | Epoch AI | |
| LiveCodeBench | 35.1% | #55 of 73 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedXpertQA-MM | 37.2% | #3 of 9 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMStar | 69.4% | #14 of 173 | Official leaderboard | |
| MMVU | 65.9% | #4 of 28 | Official leaderboard | |
| TOMATO | 33.7% | #4 of 22 | Official leaderboard | |
| CoVoST2 | 39.2 | #4 of 11 | Model card | |
| MMMU | 70.7% | #55 of 97 | Model card | |
| VISTA | 40% | #33 of 54 | Official leaderboard | |
| MMBench-EN v1.1 | 70.4% | #116 of 178 | Official leaderboard | |
| EgoSchema | 71.5% | #8 of 12 | Model card | |
| Vibe-Eval | 56.3% | #9 of 9 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AgentDojo | 17% | #12 of 20 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 72.6 | #21 of 173 | Official leaderboard | |
| LiveBench | 64.1% | #12 of 48 | Epoch AI |
The benchmarks the lab chose to headline, with the scores it gave.