Benchmark Almanac

Mercury Coder Small

Mercury Coder Small places in the top three on EvalPlus.

Inception Labs Released 26 Feb 2025 Family Mercury The lab's model card
7
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Inception Labs Feb 2025

Mercury Coder Small

Closed weights Flagship

No Epoch Capabilities Index score yet
  • BigCodeBench 45.5% #7 /11
  • LiveCodeBench 25% #64 /73
  • HumanEval 90% #13 /95

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Coding

BenchmarkScoreRankSource
HumanEval 90% #13 of 95 Model card
MBPP 76.6% #26 of 66 Model card
MultiPL-E 76.2% #8 of 18 Model card
EvalPlus 80.4% #3 of 6 Model card
BigCodeBench 45.5% #7 of 11 Model card
LiveCodeBench 25% #64 of 73 Model card
HumanEval Infilling 84.8% #1 of 1 Model card

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.