BitNet b1.58 2B4T.
Microsoft Apr 2025 Open weights Small
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BoolQ | 80.2% | #16 of 28 | Model card | |
| WinoGrande | 71.9% | #32 of 42 | Model card | |
| CommonSenseQA | 71.6% | #13 of 17 | Model card | |
| TruthfulQA | 45.3% | #22 of 28 | Model card | |
| ARC-Challenge | 49.9% | #40 of 45 | Model card | |
| ARC-Easy | 74.8% | #21 of 23 | Model card | |
| PIQA | 77.1% | #31 of 33 | Model card | |
| MMLU | 53.2% | #120 of 127 | Model card | |
| HellaSwag | 68.4% | #45 of 47 | Model card | |
| IFEval | 53.5% | #125 of 126 | Model card | |
| TriviaQA | 33.6% | #39 of 39 | Model card | |
| MT-Bench | 5.85 | #23 of 23 | Model card | |
| OpenBookQA | 41.6% | #20 of 20 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval+ | 38.4% | #27 of 27 | Model card |
The benchmarks the lab chose to headline, with the scores it gave.
Every result in the longer documents the lab published with it.