Benchmark Almanac

Code Interpreter Benchmark General

GPT-4 (snapshot unspecified) 82.8% tops its leaderboard Qwen-14B 65.5% leads the models we track
Not in model cards

Models use a Python interpreter for general tasks. The score is the percentage of generated programs that execute.

One track of Code Interpreter Benchmark. The Code Interpreter Benchmark record keeps lab scores that combine tracks.

Made by
Qwen
Released
September 2023
Kind
Function calling
Level
School
Scored
Checked automatically
Answers
Agent task
Tasks
All public
Strongest model tested
Qwen-14B #213 on the capability index
Last model added
October 2023
Scores refreshed
November 12th, 2026
10
Entries on its own leaderboard, 2 models we track

GPT-4 (snapshot unspecified) tops the leaderboard with 82.8%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen-14B, at 65.5%.

  1. 82.8 GPT-4 (snapshot unspecified)
  2. 82.8 Qwen-72B-Chat
  3. 74.1 CodeLLaMA-13B-Instruct
  4. 74.1 GPT-3.5
  5. 67.2 Qwen-7B-Chat
  6. 65.5 InternLM-20B-Chat
  7. 65.5 Qwen-1.8B-Chat
  8. 65.5 Qwen-14B
  9. 62.1 ChatGLM3-6B
  10. 48.3 LLaMA2-13B-Chat
  1. 1 GPT-4 (snapshot unspecified) version 20231206 82.8%
  2. 2 Qwen-72B-Chat version 20231206 82.8%
  3. 3 CodeLLaMA-13B-Instruct version 20231206 74.1%
  4. 4 GPT-3.5 version 20231206 74.1%
  5. 5 Qwen-7B-Chat version 20231206 67.2%
  6. 6 InternLM-20B-Chat version 20231206 65.5%
  7. 7 Qwen-1.8B-Chat version 20231206 65.5%
  8. 8 Qwen-14B version 20231206 65.5%
  9. 9 ChatGLM3-6B version 20231206 62.1%
  10. 10 LLaMA2-13B-Chat version 20231206 48.3%

Source: Official Code Interpreter Benchmark General leaderboard

#EntryScoreSettingDateSource
1 GPT-4 (snapshot unspecified) 82.8% version 20231206 Official leaderboard
2 Qwen-72B-Chat 82.8% version 20231206 Official leaderboard
3 CodeLLaMA-13B-Instruct 74.1% version 20231206 Official leaderboard
4 GPT-3.5 74.1% version 20231206 Official leaderboard
5 Qwen-7B-Chat 67.2% version 20231206 Official leaderboard
6 InternLM-20B-Chat 65.5% version 20231206 Official leaderboard
7 Qwen-1.8B-Chat 65.5% version 20231206 Official leaderboard
8 Qwen-14B 65.5% version 20231206 Sep 2023 Official leaderboard
9 ChatGLM3-6B 62.1% version 20231206 Oct 2023 Official leaderboard
10 LLaMA2-13B-Chat 48.3% version 20231206 Official leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0% 20% 40% 60% 80% 100% 2024 Near the ceiling (85%) Qwen-14B 0% 20% 40% 60% 80% 100% 2024 Near the ceiling (85%) Qwen-14B

Source: Official Code Interpreter Benchmark General leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Tool use

All 144 in Tool use
  • AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state. Gemini 4 Carbon 53.9%
  • MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks. Gemini 4 Argon 90.7%
  • BFCL V4 In use Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes. Qwen3.7-Max 75%
  • τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state. Qwen 3.8 Max 55.1%
  • Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
  • DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%