LeetCode Weekly Contest

Models solve 126 LeetCode contest problems from July to November 2023. A solution counts only if it passes all of its 20+ test cases.

Made by DeepSeek Released Nov 2023 Competitive programming Last in a launch post Nov 2023 , dropped before it saturated Last model added Jan 2024

Level: University Undergraduates and entry-level professionals
Scored: Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers: Code A program or patch run against tests.

deepseek-ai/DeepSeek-LLM 7,262 stars Paper

Model scored in lab cards

Launch post lists it, from 1 lab

System card reports it, from 1 lab

DeepSeek LLM 67B leads with 17.5%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Bars Table

17.5 DeepSeek LLM 67B

1 DeepSeek LLM 67B 17.5%

#	Model	Score	Setting	Date	Source
1	DeepSeek LLM 67B	17.5%		Jan 2024	Model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card

No launch post has listed it since Nov 2023

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

DeepSeek

Date	Launch post	Score	As the post writes it
29 Nov 2023	DeepSeek LLM 67B		LeetCode Weekly Contest (pass@1)

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
5 Jan 2024	DeepSeek LLM 67B technical report	17.5	LeetCode Weekly Contest 351-372 and Bi-Weekly 108-117 (Jul to Nov 2023), pass@1, held-out (Table 9)	Capabilities

DeepSeek LLM 67B leads with 17.5%

Scores over time

No launch post has listed it since Nov 2023

One system card reports it

More in Coding