SciBench
Llama 3-8B 8.45% leads the models we track Models solve open-ended college-level problems in mathematics, chemistry and physics. Most answers are numbers, which a script checks against the key.
The paper also sorts model errors into ten problem-solving skills through a user study. About 140 GitHub stars, and the repo was last pushed in 2024-07.
- Made by
- University of California, Los Angeles
- Released
- July 2023
- Kind
- Science questions
- Level
- University
- Scored
- Checked automatically
- Answers
- Short answer
- Tasks
- All public
- Strongest model tested
- Llama 3-8B #205 on the capability index
- Last model added
- April 2024
- Scores refreshed
- November 12th, 2026
In the authors’ words
Most of the existing Large Language Model (LLM) benchmarks on scientific problem reasoning focus on problems grounded in high-school subjects and are confined to elementary algebraic operations. To systematically examine the reasoning capabilities required for solving complex scientific problems, we introduce an expansive benchmark suite SciBench for LLMs. SciBench contains a carefully curated dataset featuring a range of collegiate-level scientific problems from mathematics, chemistry, and physics domains. Based on the dataset, we conduct an in-depth benchmarking study of representative open-source and proprietary LLMs with various prompting strategies. The results reveal that the current LLMs fall short of delivering satisfactory performance, with the best overall score of merely 43.22%. Furthermore, through a detailed user study, we categorize the errors made by LLMs into ten problem-solving abilities. Our analysis indicates that no single prompting strategy significantly outperforms the others and some strategies that demonstrate improvements in certain problem-solving skills could result in declines in other skills. We envision that SciBench will catalyze further developments in the reasoning abilities of LLMs, thereby ultimately contributing to scientific research and discovery.
GPT-4-Turbo (snapshot unspecified) tops the leaderboard with 49%
Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Llama 3-8B, at 8.45%.
- 1 GPT-4-Turbo (snapshot unspecified) Zero-Shot Learning 49%
- 2 GPT-4 (snapshot unspecified) Zero-Shot Learning 36.5%
- 3 GPT-3.5-Turbo (snapshot unspecified) Zero-Shot Learning 23.6%
- 4 Llama 3-8B Zero-Shot Learning 8.45%
- 5 Mistral-7B (snapshot unspecified) Zero-Shot Learning 8.10%
- 6 Llama 2-7B Zero-Shot Learning 1.03%
Source: Official SciBench leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-4-Turbo (snapshot unspecified) | 49% | Zero-Shot Learning | Official leaderboard | |
| 2 | GPT-4 (snapshot unspecified) | 36.5% | Zero-Shot Learning | Official leaderboard | |
| 3 | GPT-3.5-Turbo (snapshot unspecified) | 23.6% | Zero-Shot Learning | Official leaderboard | |
| 4 | Llama 3-8B | 8.45% | Zero-Shot Learning | Apr 2024 | Official leaderboard |
| 5 | Mistral-7B (snapshot unspecified) | 8.10% | Zero-Shot Learning | Official leaderboard | |
| 6 | Llama 2-7B | 1.03% | Zero-Shot Learning | Jul 2023 | Official leaderboard |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official SciBench leaderboard
No lab model card reports it yet
We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.
More in Science and reasoning
- GPQA Diamond In use 198 graduate-level 'Google-proof' multiple-choice questions in biology, chemistry and physics.
Gemini 4 Carbon 96.5% - Terminal-Bench-Science In use 70 scientific research-workflow tasks across five science areas, run in Harbor containers. Hidden tests give the resolution rate. GPT-6 Astra 68.1%
- FrontierScience In use Models solve physics, chemistry, and biology problems in an Olympiad track graded by short answer and a Research track graded by 10-point rubrics.
Intern-S2-397B 87% - BioMysteryBench In use Bioinformatics problems on anonymized real datasets that require real analysis with tools. Rubric-graded, with human-solvable and human-difficult splits. Claude Fable 5.2 83.7%
- HiPhO In use Models solve theory problems from 2024 and 2025 physics olympiad exams, some with diagrams. Official marking schemes grade answers and steps, mapped to medal levels. Gemini 3 Pro 78.3%
- LABBench2 In use 11 biology research sub-tasks (literature, figures, tables, patents, trials and more) with tool access. Macro-averaged accuracy. GPT-5.2 60.9%