- MMLU-Pro 71.6
- LiveCodeBench (v5) (UI 10/01/2024-02/01/2025) 28.9
- Bird-SQL (Dev) 57.4
- GPQA (diamond) 51.5
- SimpleQA 21.7
- FACTS Grounding 83.6
- Global MMLU (Lite) 78.2
- MATH 86.8
- HiddenMath 55.3
- MRCR (1M) 58
- MMMU 68
- CoVoST2 (21 lang), BLEU 38.4
- EgoSchema (test) 67.2
Gemini 2.0 Flash-Lite
Gemini 2.0 Flash-Lite.
Google DeepMind Released 5 Feb 2025 36
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Google DeepMind Feb 2025 Gemini 2.0 Flash-Lite
Closed weights
No Epoch Capabilities Index score yet
- LiveCodeBench v5 28.9% #41 /44
- Bird-SQL 57.4% #6 /13
- Vectara Hallucination Leaderboard (HHEM) 1.20% #142 /148
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BIG-Bench Extra Hard | 19.7% | #5 of 11 | Official leaderboard | |
| LMArena (Elo) | 1,354 | #172 of 333 | Official leaderboard | |
| FACTS Grounding | 83.6% | #9 of 15 | Model card | |
| Global MMLU Lite | 78.2% | #10 of 16 | Model card | |
| SimpleQA | 21.7% | #37 of 59 | Model card | |
| MMLU-Pro | 71.6% | #108 of 151 | Model card | |
| MRCR (Michelangelo, original) | 58% | #7 of 9 | Model card | |
| Vectara Hallucination Leaderboard (HHEM) | 1.20% | #142 of 148 | Official leaderboard | |
| Phare | 57.3% | #60 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 86.8% | #12 of 93 | Model card | |
| ForecastBench | 56.9 | #15 of 86 | Official leaderboard | |
| HiddenMath | 55.3% | #6 of 11 | Model card | |
| GPQA Diamond | 51.5% | #226 of 284 | Model card | |
| DTBench | 20.9% | #154 of 178 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Bird-SQL | 57.4% | #6 of 13 | Model card | |
| BigCodeBench-Hard | 25.4% | #46 of 96 | Official leaderboard | |
| LiveCodeBench v5 | 28.9% | #41 of 44 | Model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Galileo Agent Leaderboard | 0.93 | #4 of 31 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 56.4% | #36 of 67 | Official leaderboard | |
| SnakeBench | 21 | #110 of 183 | Official leaderboard | |
| ToolComp | 53.8% | #20 of 32 | Official leaderboard | |
| LisanBench | 157.3 | #97 of 133 | Official leaderboard | |
| LLM Chess | -812 | #122 of 124 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMMU | 68% | #39 of 136 | Official leaderboard | |
| VHELM | 59.7 | #16 of 41 | Official leaderboard | |
| AHELM | 54.0 | #4 of 10 | Official leaderboard | |
| CoVoST2 | 38.4 | #7 of 11 | Model card | |
| Vision Arena | 1,136 | #106 of 136 | Official leaderboard | |
| VISTA | 32.7% | #45 of 54 | Official leaderboard | |
| EgoSchema | 67.2% | #11 of 12 | Model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Anthropic Red Team | 99.3% | #25 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 67.5% | #38 of 82 | Official leaderboard | |
| HELM Safety | 90.8 | #42 of 75 | Official leaderboard | |
| SimpleSafetyTests | 97.7% | #50 of 75 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 54.3% | #21 of 48 | Epoch AI | |
| HELM Capabilities | 64.2 | #34 of 60 | Official leaderboard |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 5 of them aren't in the launch post.
Gemini 2.0 Flash-Lite Model Card 10 Apr 2025
Capabilities
- MMLU-Pro 71.6
- LiveCodeBench v5 28.9
- Bird-SQL 57.4
- GPQA Diamond 51.5
- SimpleQA 21.7
- FACTS Grounding 83.6
- Global MMLU Lite 78.2
- MATH 86.8
- HiddenMath 55.3
- MRCR (Michelangelo, original) 58
- MMMU 68
- CoVoST2 38.4
- EgoSchema 67.2