Benchmark Almanac

IFEval

Gemma 4 31B IT 98.9% leads since Jul 2026 In use

Models answer about 500 prompts containing verifiable instructions such as word limits or required keywords. Code checks each instruction for strict and loose accuracy.

Frontier models now score near 90 percent, so it is close to saturation.

In the authors’ words

One core capability of Large Language Models (LLMs) is to follow natural language instructions. However, the evaluation of such abilities is not standardized: Human evaluations are expensive, slow, and not objectively reproducible, while LLM-based auto-evaluation is potentially biased or limited by the ability of the evaluator LLM. To overcome these issues, we introduce Instruction-Following Eval (IFEval) for large language models. IFEval is a straightforward and easy-to-reproduce evaluation benchmark. It focuses on a set of "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times". We identified 25 types of those verifiable instructions and constructed around 500 prompts, with each prompt containing one or more verifiable instructions. We show evaluation results of two widely available LLMs on the market. Our code and data can be found at https://github.com/google-research/google-research/tree/master/instruction_following_eval

Abstract of Instruction-Following Evaluation for Large Language Models (arXiv:2311.07911, CC0)
Made by Google Released Nov 2023 Following precise instructions All tasks public Last in a launch post Oct 2026 Last model added Oct 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
google-research/google-research 39,080 stars Paper Dataset
126
Models scored in lab cards
94
Launch posts list it, from 33 labs
86
System cards report it, from 31 labs

Gemma 4 31B IT leads with 98.9%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

126 models. Scroll sideways for the rest.

  1. 98.9 Gemma 4 31B IT
  2. 98.5 Gemma 4 26B A4B
  3. 97.4 DiffusionGemma 26B A4B
  4. 96.8 LFM2.5-24B-A2B
  5. 96.7 LFM3-1.7B
  6. 94.6 Qwen3.7-Plus
  7. 94.3 Qwen 3.6 Plus
  8. 94.3 Qwen3.7-Max
  9. 93.8 Olmo 3.1 32B Think
  10. 93.6 Spark-X2.6-4B
  11. 93.4 Qwen3.5-122B-A10B
  12. 93.3 ERNIE 5.0
  13. 93.2 Claude 3.7 Sonnet
  14. 92.6 Qwen3.5 397B-A17B
  15. 92.4 K-EXAONE 2.0
  16. 92.2 Keye-VL-2.0-30B-A3B
  17. 92.1 Llama 3.3 70B Instruct
  18. 91.8 LFM2.5-8B-A1B
  19. 91.5 Amazon Nova Premier
  20. 90.9 Cohere Command A
  21. 90.4 Gemma 3 27B
  22. 90.3 nvidia-nemotron-nano-9b-v2
  23. 90.2 Claude 3.5 Sonnet (October 2024)
  24. 90.2 Gemma 3 4B
  25. 90.0 LongCat-2.0
  26. 89.8 Kimi K2
  27. 89.7 K-EXAONE
  28. 89.7 Qwen3.5-Omni-Plus
  29. 89.7 LongCat-Flash-Chat
  30. 89.7 Granite 4.1 30B
  31. 89.6 EXAONE 4.5 33B
  32. 89.5 Llama-3.1-Nemotron-Ultra-253B-v1
  33. 89.4 Mistral Medium 3
  34. 89.4 Falcon-H1-34B
  35. 89.2 Llama-3.3-Nemotron-Super-49B-v1
  36. 89.1 MiniMax-Text-01
  37. 89.1 Apple Foundation Model Server (2025)
  38. 89.0 Olmo 3 32B Think
  39. 88.9 Gemma 3 12B
  40. 88.9 Qwen3-Next-80B-A3B
  41. 88.6 Llama-3.3-Nemotron-Super-49B-v1.5
  42. 88.6 Llama 3.1-405B
  43. 88.6 Hunyuan-TurboS
  44. 88.4 LFM2.5-1.2B-Thinking
  45. 88.2 Qwen3-VL-235B-A22B
  46. 88.0 ERNIE-4.5-300B-A47B
  47. 88.0 Solar Open 100B
  48. 87.9 Nemotron 3 Nano 4B
  49. 87.8 Solar Pro 2
  50. 87.8 Qwen3-235B-A22B-Thinking
  51. 87.5 Granite 4.0 H Small
  52. 87.5 Llama 3.1-70B
  53. 87.4 Seed-Thinking-v1.5
  54. 87.4 GPT-4.1
  55. 87.2 Kimi k1.5
  56. 87.0 Sarvam-M
  57. 86.9 LongCat-Flash-Thinking
  58. 86.8 sonar
  59. 86.7 Intern-S1
  60. 86.5 Qwen3-30B-A3B
  61. 86.3 Qwen2.5-VL-72B
  62. 86.3 GLM-4.5-Air
  63. 86.2 LFM2.5-1.2B-Instruct
  64. 86.1 DeepSeek-V3
  65. 86.1 GLM-4.5
  66. 86.0 Tulu 3 405B
  67. 86.0 Baichuan-M2-32B
  68. 85.9 Claude 3.5 Haiku
  69. 85.8 seed-oss-36b-instruct
  70. 85.6 OLMo 2 32B Instruct
  71. 85.4 Qwen3-14B
  72. 85.2 Ring-1T
  73. 85.0 Qwen3-32B
  74. 85.0 Qwen3-8B
  75. 84.9 Phi-4-Reasoning-plus
  76. 84.8 EXAONE 4.0 32B
  77. 84.8 Sarvam 105B
  78. 84.7 Hunyuan-A13B-Instruct
  79. 84.5 GLM-Z1-32B-0414
  80. 84.3 Qwen2.5-14B-Instruct-1M
  81. 84.3 ERNIE-4.5-21B-A3B-Thinking
  82. 84.1 Qwen2.5-72B
  83. 84.1 GPT-4.1 mini
  84. 83.7 InternVL3.5-241B-A28B
  85. 83.4 Qwen3-235B-A22B
  86. 83.4 Phi-4-reasoning
  87. 83.3 DeepSeek-R1
  88. 82.9 Mistral Small 3
  89. 82.4 LongCat-Flash-Omni
  90. 82.3 LFM2.5-VL-3B
  91. 82.2 STEP3-VL-10B
  92. 81.9 Qwen3-4B
  93. 81.5 INTELLECT-2
  94. 81.5 Hermes 4 405B
  95. 81.0 Qwen2.5-14B
  96. 81.0 Qwen3-Omni-30B-A3B
  97. 80.4 Llama 3.1-8B
  98. 80.2 Gemma 3 1B
  99. 79.5 Qwen2.5-32B
  100. 79.3 InternLM3-8B-Instruct
  101. 78.5 Aya Vision 32B
  102. 78.2 Nemotron Nano 12B v2 VL
  103. 77.9 Hermes 4.3 36B
  104. 77.6 Qwen2-72B
  105. 77.6 LFM2-8B-A1B
  106. 77.4 granite-4.0-1b
  107. 76.3 Qwen-Turbo
  108. 75.2 Apertus 70B
  109. 74.9 North Micro Vision Instruct
  110. 74.9 LFM2-1.2B
  111. 74.8 Granite 3.3 8B Instruct
  112. 74.5 GPT-4.1 nano
  113. 74.3 Granite 3.2 8B Instruct
  114. 72.5 Qwen3-1.7B
  115. 71.2 Qwen2.5-7B
  116. 70.1 Phi-4 Mini
  117. 70.1 OLMo 2 1B Instruct
  118. 67.7 GLM-4
  119. 67.0 AFM-4.5B
  120. 63.8 DeepSeek-V2 (MoE-236B, May 2024)
  121. 63.0 Phi-4
  122. 61.0 MiMo-7B-RL
  123. 57.6 Falcon-Arabic-7B
  124. 55.5 DeepSeek LLM 67B
  125. 53.5 BitNet b1.58 2B4T
  126. 51.2 Gemma 3 270M
  1. 1 Gemma 4 31B IT 98.9%
  2. 2 Gemma 4 26B A4B 98.5%
  3. 3 DiffusionGemma 26B A4B 97.4%
  4. 4 LFM2.5-24B-A2B 96.8%
  5. 5 LFM3-1.7B 96.7%
  6. 6 Qwen3.7-Plus 94.6%
  7. 7 Qwen 3.6 Plus 94.3%
  8. 8 Qwen3.7-Max 94.3%
  9. 9 Olmo 3.1 32B Think 93.8%
  10. 10 Spark-X2.6-4B 93.6%
  11. 11 Qwen3.5-122B-A10B 93.4%
  12. 12 ERNIE 5.0 93.3%
  13. 13 Claude 3.7 Sonnet 93.2%
  14. 14 Qwen3.5 397B-A17B 92.6%
  15. 15 K-EXAONE 2.0 92.4%
  16. 16 Keye-VL-2.0-30B-A3B 92.2%
  17. 17 Llama 3.3 70B Instruct 92.1%
  18. 18 LFM2.5-8B-A1B 91.8%
  19. 19 Amazon Nova Premier 91.5%
  20. 20 Cohere Command A 90.9%
  21. 21 Gemma 3 27B 90.4%
  22. 22 nvidia-nemotron-nano-9b-v2 90.3%
  23. 23 Claude 3.5 Sonnet (October 2024) 90.2%
  24. 24 Gemma 3 4B 90.2%
  25. 25 LongCat-2.0 90%
  26. 26 Kimi K2 Jul 2025 89.8%
  27. 27 K-EXAONE 89.7%
  28. 28 Qwen3.5-Omni-Plus 89.7%
  29. 29 LongCat-Flash-Chat 89.7%
  30. 30 Granite 4.1 30B 89.7%
  31. 31 EXAONE 4.5 33B 89.6%
  32. 32 Llama-3.1-Nemotron-Ultra-253B-v1 89.5%
  33. 33 Mistral Medium 3 89.4%
  34. 34 Falcon-H1-34B 89.4%
  35. 35 Llama-3.3-Nemotron-Super-49B-v1 89.2%
  36. 36 MiniMax-Text-01 89.1%
  37. 37 Apple Foundation Model Server (2025) 89.1%
  38. 38 Olmo 3 32B Think 89%
  39. 39 Gemma 3 12B 88.9%
  40. 40 Qwen3-Next-80B-A3B 88.9%
  41. 41 Llama-3.3-Nemotron-Super-49B-v1.5 88.6%
  42. 42 Llama 3.1-405B 88.6%
  43. 43 Hunyuan-TurboS 88.6%
  44. 44 LFM2.5-1.2B-Thinking 88.4%
  45. 45 Qwen3-VL-235B-A22B 88.2%
  46. 46 ERNIE-4.5-300B-A47B 88%
  47. 47 Solar Open 100B 88%
  48. 48 Nemotron 3 Nano 4B 87.9%
  49. 49 Solar Pro 2 87.8%
  50. 50 Qwen3-235B-A22B-Thinking Jul 2025 87.8%
  51. 51 Granite 4.0 H Small 87.5%
  52. 52 Llama 3.1-70B 87.5%
  53. 53 Seed-Thinking-v1.5 87.4%
  54. 54 GPT-4.1 87.4%
  55. 55 Kimi k1.5 87.2%
  56. 56 Sarvam-M 87%
  57. 57 LongCat-Flash-Thinking 86.9%
  58. 58 sonar 86.8%
  59. 59 Intern-S1 86.7%
  60. 60 Qwen3-30B-A3B 86.5%
  61. 61 Qwen2.5-VL-72B 86.3%
  62. 62 GLM-4.5-Air 86.3%
  63. 63 LFM2.5-1.2B-Instruct 86.2%
  64. 64 DeepSeek-V3 86.1%
  65. 65 GLM-4.5 86.1%
  66. 66 Tulu 3 405B 86%
  67. 67 Baichuan-M2-32B 86%
  68. 68 Claude 3.5 Haiku 85.9%
  69. 69 seed-oss-36b-instruct 85.8%
  70. 70 OLMo 2 32B Instruct 85.6%
  71. 71 Qwen3-14B 85.4%
  72. 72 Ring-1T 85.2%
  73. 73 Qwen3-32B 85%
  74. 74 Qwen3-8B 85%
  75. 75 Phi-4-Reasoning-plus 84.9%
  76. 76 EXAONE 4.0 32B 84.8%
  77. 77 Sarvam 105B 84.8%
  78. 78 Hunyuan-A13B-Instruct 84.7%
  79. 79 GLM-Z1-32B-0414 84.5%
  80. 80 Qwen2.5-14B-Instruct-1M 84.3%
  81. 81 ERNIE-4.5-21B-A3B-Thinking 84.3%
  82. 82 Qwen2.5-72B 84.1%
  83. 83 GPT-4.1 mini 84.1%
  84. 84 InternVL3.5-241B-A28B 83.7%
  85. 85 Qwen3-235B-A22B 83.4%
  86. 86 Phi-4-reasoning 83.4%
  87. 87 DeepSeek-R1 83.3%
  88. 88 Mistral Small 3 82.9%
  89. 89 LongCat-Flash-Omni 82.4%
  90. 90 LFM2.5-VL-3B 82.3%
  91. 91 STEP3-VL-10B 82.2%
  92. 92 Qwen3-4B 81.9%
  93. 93 INTELLECT-2 81.5%
  94. 94 Hermes 4 405B 81.5%
  95. 95 Qwen2.5-14B 81%
  96. 96 Qwen3-Omni-30B-A3B 81%
  97. 97 Llama 3.1-8B 80.4%
  98. 98 Gemma 3 1B 80.2%
  99. 99 Qwen2.5-32B 79.5%
  100. 100 InternLM3-8B-Instruct 79.3%
  101. 101 Aya Vision 32B 78.5%
  102. 102 Nemotron Nano 12B v2 VL 78.2%
  103. 103 Hermes 4.3 36B 77.9%
  104. 104 Qwen2-72B 77.6%
  105. 105 LFM2-8B-A1B 77.6%
  106. 106 granite-4.0-1b 77.4%
  107. 107 Qwen-Turbo 76.3%
  108. 108 Apertus 70B 75.2%
  109. 109 North Micro Vision Instruct 74.9%
  110. 110 LFM2-1.2B 74.9%
  111. 111 Granite 3.3 8B Instruct 74.8%
  112. 112 GPT-4.1 nano 74.5%
  113. 113 Granite 3.2 8B Instruct 74.3%
  114. 114 Qwen3-1.7B 72.5%
  115. 115 Qwen2.5-7B 71.2%
  116. 116 Phi-4 Mini 70.1%
  117. 117 OLMo 2 1B Instruct 70.1%
  118. 118 GLM-4 67.7%
  119. 119 AFM-4.5B 67%
  120. 120 DeepSeek-V2 (MoE-236B, May 2024) 63.8%
  121. 121 Phi-4 63%
  122. 122 MiMo-7B-RL 61%
  123. 123 Falcon-Arabic-7B 57.6%
  124. 124 DeepSeek LLM 67B 55.5%
  125. 125 BitNet b1.58 2B4T 53.5%
  126. 126 Gemma 3 270M 51.2%
#ModelScoreSettingDateSource
1 Gemma 4 31B IT 98.9% Jul 2026 Model card
2 Gemma 4 26B A4B 98.5% Jul 2026 Model card
3 DiffusionGemma 26B A4B 97.4% Jul 2026 Model card
4 LFM2.5-24B-A2B 96.8% Oct 2026 Model card
5 LFM3-1.7B 96.7% Oct 2026 Model card
6 Qwen3.7-Plus 94.6% Jun 2026 Model card
7 Qwen 3.6 Plus 94.3% Apr 2026 Model card
8 Qwen3.7-Max 94.3% May 2026 Model card
9 Olmo 3.1 32B Think 93.8% Dec 2025 Model card
10 Spark-X2.6-4B 93.6% Oct 2026 Model card
11 Qwen3.5-122B-A10B 93.4% Feb 2026 Model card
12 ERNIE 5.0 93.3% Feb 2026 Model card
13 Claude 3.7 Sonnet 93.2% Feb 2025 Model card
14 Qwen3.5 397B-A17B 92.6% Feb 2026 Model card
15 K-EXAONE 2.0 92.4% Jul 2026 Model card
16 Keye-VL-2.0-30B-A3B 92.2% May 2026 Model card
17 Llama 3.3 70B Instruct 92.1% Dec 2024 Model card
18 LFM2.5-8B-A1B 91.8% May 2026 Model card
19 Amazon Nova Premier 91.5% Apr 2025 Model card
20 Cohere Command A 90.9% Apr 2025 Model card
21 Gemma 3 27B 90.4% Mar 2025 Model card
22 nvidia-nemotron-nano-9b-v2 90.3% Aug 2025 Model card
23 Claude 3.5 Sonnet (October 2024) 90.2% Oct 2024 Model card
24 Gemma 3 4B 90.2% Mar 2025 Model card
25 LongCat-2.0 90% Jul 2026 Model card
26 Kimi K2 Jul 2025 89.8% Jul 2025 Model card
27 K-EXAONE 89.7% Dec 2025 Model card
28 Qwen3.5-Omni-Plus 89.7% Mar 2026 Model card
29 LongCat-Flash-Chat 89.7% Aug 2025 Model card
30 Granite 4.1 30B 89.7% Apr 2026 Model card
31 EXAONE 4.5 33B 89.6% Apr 2026 Model card
32 Llama-3.1-Nemotron-Ultra-253B-v1 89.5% Apr 2025 Model card
33 Mistral Medium 3 89.4% May 2025 Model card
34 Falcon-H1-34B 89.4% May 2025 Model card
35 Llama-3.3-Nemotron-Super-49B-v1 89.2% Mar 2025 Model card
36 MiniMax-Text-01 89.1% Jan 2025 Model card
37 Apple Foundation Model Server (2025) 89.1% Jul 2025 Model card
38 Olmo 3 32B Think 89% Nov 2025 Model card
39 Gemma 3 12B 88.9% Mar 2025 Model card
40 Qwen3-Next-80B-A3B 88.9% Sep 2025 Model card
41 Llama-3.3-Nemotron-Super-49B-v1.5 88.6% Jul 2025 Model card
42 Llama 3.1-405B 88.6% Jul 2024 Model card
43 Hunyuan-TurboS 88.6% Feb 2025 Model card
44 LFM2.5-1.2B-Thinking 88.4% Jan 2026 Model card
45 Qwen3-VL-235B-A22B 88.2% Sep 2025 Model card
46 ERNIE-4.5-300B-A47B 88% Jun 2025 Model card
47 Solar Open 100B 88% Dec 2025 Model card
48 Nemotron 3 Nano 4B 87.9% Mar 2026 Model card
49 Solar Pro 2 87.8% Jul 2025 Model card
50 Qwen3-235B-A22B-Thinking Jul 2025 87.8% Jul 2025 Model card
51 Granite 4.0 H Small 87.5% Oct 2025 Model card
52 Llama 3.1-70B 87.5% Jul 2024 Model card
53 Seed-Thinking-v1.5 87.4% Apr 2025 Model card
54 GPT-4.1 87.4% Apr 2025 Model card
55 Kimi k1.5 87.2% Jan 2025 Model card
56 Sarvam-M 87% May 2025 Model card
57 LongCat-Flash-Thinking 86.9% Sep 2025 Model card
58 sonar 86.8% Feb 2025 Model card
59 Intern-S1 86.7% Jul 2025 Model card
60 Qwen3-30B-A3B 86.5% May 2025 Model card
61 Qwen2.5-VL-72B 86.3% Feb 2025 Model card
62 GLM-4.5-Air 86.3% Aug 2025 Model card
63 LFM2.5-1.2B-Instruct 86.2% Jan 2026 Model card
64 DeepSeek-V3 86.1% Dec 2024 Model card
65 GLM-4.5 86.1% Aug 2025 Model card
66 Tulu 3 405B 86% Jan 2025 Model card
67 Baichuan-M2-32B 86% Sep 2025 Model card
68 Claude 3.5 Haiku 85.9% Oct 2024 Model card
69 seed-oss-36b-instruct 85.8% Aug 2025 Model card
70 OLMo 2 32B Instruct 85.6% Mar 2025 Model card
71 Qwen3-14B 85.4% May 2025 Model card
72 Ring-1T 85.2% Oct 2025 Model card
73 Qwen3-32B 85% May 2025 Model card
74 Qwen3-8B 85% May 2025 Model card
75 Phi-4-Reasoning-plus 84.9% Apr 2025 Model card
76 EXAONE 4.0 32B 84.8% Jul 2025 Model card
77 Sarvam 105B 84.8% Mar 2026 Model card
78 Hunyuan-A13B-Instruct 84.7% Jun 2025 Model card
79 GLM-Z1-32B-0414 84.5% Apr 2025 Model card
80 Qwen2.5-14B-Instruct-1M 84.3% Jan 2025 Model card
81 ERNIE-4.5-21B-A3B-Thinking 84.3% Sep 2025 Model card
82 Qwen2.5-72B 84.1% Sep 2024 Model card
83 GPT-4.1 mini 84.1% Apr 2025 Model card
84 InternVL3.5-241B-A28B 83.7% Aug 2025 Model card
85 Qwen3-235B-A22B 83.4% Apr 2025 Model card
86 Phi-4-reasoning 83.4% Apr 2025 Model card
87 DeepSeek-R1 83.3% Jan 2025 Model card
88 Mistral Small 3 82.9% Jan 2025 Model card
89 LongCat-Flash-Omni 82.4% Oct 2025 Model card
90 LFM2.5-VL-3B 82.3% Aug 2026 Model card
91 STEP3-VL-10B 82.2% Jan 2026 Model card
92 Qwen3-4B 81.9% May 2025 Model card
93 INTELLECT-2 81.5% May 2025 Model card
94 Hermes 4 405B 81.5% Aug 2025 Model card
95 Qwen2.5-14B 81% Dec 2024 Model card
96 Qwen3-Omni-30B-A3B 81% Sep 2025 Model card
97 Llama 3.1-8B 80.4% Jul 2024 Model card
98 Gemma 3 1B 80.2% Mar 2025 Model card
99 Qwen2.5-32B 79.5% Dec 2024 Model card
100 InternLM3-8B-Instruct 79.3% Jan 2025 Model card
101 Aya Vision 32B 78.5% May 2025 Model card
102 Nemotron Nano 12B v2 VL 78.2% Nov 2025 Model card
103 Hermes 4.3 36B 77.9% Dec 2025 Model card
104 Qwen2-72B 77.6% Jun 2024 Model card
105 LFM2-8B-A1B 77.6% Oct 2025 Model card
106 granite-4.0-1b 77.4% Oct 2025 Model card
107 Qwen-Turbo 76.3% Dec 2024 Model card
108 Apertus 70B 75.2% Sep 2025 Model card
109 North Micro Vision Instruct 74.9% Aug 2026 Model card
110 LFM2-1.2B 74.9% Jul 2025 Model card
111 Granite 3.3 8B Instruct 74.8% Apr 2025 Model card
112 GPT-4.1 nano 74.5% Apr 2025 Model card
113 Granite 3.2 8B Instruct 74.3% Feb 2025 Model card
114 Qwen3-1.7B 72.5% May 2025 Model card
115 Qwen2.5-7B 71.2% Dec 2024 Model card
116 Phi-4 Mini 70.1% Mar 2025 Model card
117 OLMo 2 1B Instruct 70.1% Apr 2025 Model card
118 GLM-4 67.7% Jan 2024 Model card
119 AFM-4.5B 67% Jul 2025 Model card
120 DeepSeek-V2 (MoE-236B, May 2024) 63.8% May 2024 Model card
121 Phi-4 63% Dec 2024 Model card
122 MiMo-7B-RL 61% Apr 2025 Model card
123 Falcon-Arabic-7B 57.6% Nov 2025 Model card
124 DeepSeek LLM 67B 55.5% Jan 2024 Model card
125 BitNet b1.58 2B4T 53.5% Apr 2025 Model card
126 Gemma 3 270M 51.2% Aug 2025 Model card

11 models have held number one since its release in Nov 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

DeepSeek Z.ai Qwen Meta Anthropic Ai2 Google DeepMind
Lab claims Passed 85% in Jul 2024, by Llama 3.1-405B
Gemma 4 31B IT 98.9%

The best score passed 85% in Jul 2024

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT Qwen3.7-Plus Olmo 3.1 32B Think Claude 3.7 Sonnet Llama 3.1-405B Qwen2-72B GLM-4 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Gemma 4 31B IT Llama 3.3 70B Instruct

6 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

DeepSeek
Z.ai
Qwen
Meta
Anthropic
Microsoft
Shanghai AI Lab
MiniMax
Moonshot AI
Mistral AI
Ai2
Perplexity
IBM
Tencent
Google DeepMind
Cohere
NVIDIA
ByteDance Seed
OpenAI
Xiaomi Corp
Amazon
Prime Intellect
Technology Innovation Institute
Sarvam AI
Baidu
Upstage
Liquid AI
LG AI Research
Arcee AI
Nous Research
Meituan
Kuaishou
iFlytek
DateLaunch postScoreAs the post writes it
27 Oct 2026 LFM3-1.7B 96.7 IFEval
22 Oct 2026 Spark-X2.6-4B 93.6 IFEval
13 Oct 2026 LFM2.5-24B-A2B 96.8 IFEval
12 Aug 2026 LFM2.5-VL-3B 82.3 IFEval
12 Aug 2026 North Micro Vision Instruct 74.9 IFEval
31 Jul 2026 K-EXAONE 2.0 92.4 IFEval
5 Jul 2026 LongCat-2.0 90 IFEval
1 Jun 2026 Qwen3.7-Plus 94.6 IFEval
28 May 2026 LFM2.5-8B-A1B 91.8 IFEval
25 May 2026 Keye-VL-2.0-30B-A3B 92.2 IFEval
20 May 2026 Qwen3.7-Max 94.3 IFEval
29 Apr 2026 Granite 4.1 30B 89.7 IFEval Avg
9 Apr 2026 EXAONE 4.5 33B 89.6 IFEval
2 Apr 2026 Qwen 3.6 Plus 94.3 IFEvalstrict prompt
30 Mar 2026 Qwen3.5-Omni-Plus 89.7 IFEval
16 Mar 2026 Nemotron 3 Nano 4B 87.9 IFEVAL-Prompt (reasoning on); Instruction 92
6 Mar 2026 Sarvam 105B 84.8 IF Eval
24 Feb 2026 Qwen3.5-122B-A10B 93.4 IFEval
16 Feb 2026 Qwen3.5 397B-A17B 92.6 IFEval
6 Feb 2026 ERNIE 5.0 93.3 IFEval
20 Jan 2026 LFM2.5-1.2B-Thinking 88.4 IFEval (± 0.35)
5 Jan 2026 LFM2.5-1.2B-Instruct 86.2 IFEval
31 Dec 2025 Solar Open 100B 88 IFEval
31 Dec 2025 K-EXAONE 89.7 IFEval
12 Dec 2025 Olmo 3.1 32B Think 93.8 IFEval
3 Dec 2025 Hermes 4.3 36B 77.9 IFEval
20 Nov 2025 Olmo 3 32B Think 89 IFEval
31 Oct 2025 LongCat-Flash-Omni 82.4 IFEval(acc), text
28 Oct 2025 granite-4.0-1b 77.4 IFEval Average
7 Oct 2025 LFM2-8B-A1B 77.6 IFEval
2 Oct 2025 Granite 4.0 H Small 87.5 IFEval Average
23 Sep 2025 Qwen3-VL-235B-A22B 88.2 IFEval
22 Sep 2025 Qwen3-Omni-30B-A3B 81 IFEval
21 Sep 2025 LongCat-Flash-Thinking 86.9 IFEval(strict prompt)
11 Sep 2025 Qwen3-Next-80B-A3B 88.9 IFEval
9 Sep 2025 ERNIE-4.5-21B-A3B-Thinking 84.3 IFEval (Prompt Strict Accuracy)
29 Aug 2025 LongCat-Flash-Chat 89.7 IFEval(acc)
26 Aug 2025 Hermes 4 405B 81.5 IFEval (Loose) (reasoning)
25 Aug 2025 InternVL3.5-241B-A28B 83.7 IFEval
20 Aug 2025 seed-oss-36b-instruct 85.8 IFEval
18 Aug 2025 nvidia-nemotron-nano-9b-v2 90.3 IFEval (Instruction Strict)
14 Aug 2025 Gemma 3 270M 51.2 IF Eval 0-shot (IT)
29 Jul 2025 AFM-4.5B 67 IFEval Inst Strict
25 Jul 2025 Llama-3.3-Nemotron-Super-49B-v1.5 88.6 IFEval (Strict:Instruction)
25 Jul 2025 Qwen3-235B-A22B-Thinking 87.8 IFEval
24 Jul 2025 Intern-S1 86.7 IFEval
15 Jul 2025 EXAONE 4.0 32B 84.8 IFEval 84.8 non-reasoning (83.7 reasoning)
11 Jul 2025 Kimi K2 89.8 IFEval, Prompt Strict
10 Jul 2025 LFM2-1.2B 74.9 IFEval
10 Jul 2025 Solar Pro 2 87.8 IFEval
30 Jun 2025 ERNIE-4.5-300B-A47B 88 IFEval
27 Jun 2025 Hunyuan-A13B-Instruct 84.7 IF-Eval
23 May 2025 Sarvam-M 87 IFEval (0.87)
20 May 2025 Falcon-H1-34B 89.4 IFEval
11 May 2025 INTELLECT-2 81.5 IFEval
7 May 2025 Mistral Medium 3 89.4 IfEval 0-shot
30 Apr 2025 Phi-4-reasoning 83.4 IFEval Strict
30 Apr 2025 Amazon Nova Premier 91.5 IFEval
30 Apr 2025 Phi-4-Reasoning-plus 84.9 IFEval Strict
29 Apr 2025 OLMo 2 1B Instruct 70.1 IFEval
29 Apr 2025 MiMo-7B-RL 61 IF-Eval (Prompt Strict)
29 Apr 2025 Qwen3-235B-A22B 83.4 IFEval strict prompt
16 Apr 2025 Granite 3.3 8B Instruct 74.8 IFEval
15 Apr 2025 BitNet b1.58 2B4T 53.5 IFEval
14 Apr 2025 GPT-4.1 nano 74.5 IFEval
14 Apr 2025 GPT-4.1 mini 84.1 IFEval
14 Apr 2025 GLM-Z1-32B-0414 84.5 IFEval
14 Apr 2025 GPT-4.1 87.4 IFEval
10 Apr 2025 Seed-Thinking-v1.5 87.4 IFEval
7 Apr 2025 Llama-3.1-Nemotron-Ultra-253B-v1 89.5 IFEval Strict:Instruction (reasoning on)
18 Mar 2025 Llama-3.3-Nemotron-Super-49B-v1 89.2 IFEval Strict:Instruction (reasoning off)
13 Mar 2025 Cohere Command A IFEval (chart only)
13 Mar 2025 OLMo 2 32B Instruct 85.6 IFEval
12 Mar 2025 Gemma 3 27B 90.4 IFEval 0-shot (IT)
6 Mar 2025 QwQ-32B IFEval
27 Feb 2025 Hunyuan-TurboS 88.6 IF-Eval
26 Feb 2025 Granite 3.2 8B Instruct 74.3 IFEval
24 Feb 2025 Claude 3.7 Sonnet 93.2 Instruction-following IFEval (64K extended thinking)
11 Feb 2025 sonar 86.8 IFEval
30 Jan 2025 Tulu 3 405B 86 IFEval (prompt loose)
30 Jan 2025 Mistral Small 3 82.9 ifeval
20 Jan 2025 Kimi k1.5 87.2 IF-Eval (Prompt Strict), short-CoT
20 Jan 2025 DeepSeek-R1 83.3 IF-Eval (Prompt Strict)
15 Jan 2025 MiniMax-Text-01 89.1 IFEval (avg)
13 Jan 2025 InternLM3-8B-Instruct 79.3 IFEval(Prompt-Strict)
26 Dec 2024 DeepSeek-V3 86.1 IF-Eval (Prompt Strict)
12 Dec 2024 Phi-4 63 IFEval
6 Dec 2024 Llama 3.3 70B Instruct 92.1 IFEval
22 Oct 2024 Claude 3.5 Sonnet (October 2024) 90.2 IFEval
19 Sep 2024 Qwen2.5-72B 84.1 IFeval strict-prompt
23 Jul 2024 Llama 3.1-405B 88.6 IFEval
7 Jun 2024 Qwen2-72B 77.6 IFEval (Prompt Strict-Acc.)
16 Jan 2024 GLM-4 67.7 IFEval Prompt级别、英文 (prompt-level, English)
29 Nov 2023 DeepSeek LLM 67B Instruction Following Evaluation (prompt-level loose)

86 system cards report it, from 31 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
5 Aug 2026 K-EXAONE 2.0 technical report 92.4 IFEval (Table 6, Reasoning mode) Capabilities
31 Jul 2026 DiffusionGemma 26B A4B technical report 97.4 IFEval, TD, 94.5 no-think (Table 3) Capabilities
24 Jul 2026 Gemma 4 31B IT technical report 98.9 IFEval, thinking (Table 5) Capabilities
24 Jul 2026 Gemma 4 26B A4B technical report 98.5 IFEval, thinking (Table 5) Capabilities
9 Jun 2026 Keye-VL-2.0-30B-A3B technical report 92.2 IFEval (Figure 5, printed value) Capabilities
17 Apr 2026 Qwen3.5-Omni-Plus technical report 89.7 IFEval (Table 4) Capabilities
9 Apr 2026 EXAONE 4.5 33B technical report 89.6 IFEval (Table 3, Reasoning mode) Capabilities
4 Feb 2026 ERNIE 5.0 technical report 93.3 IFEval (Table 2) Capabilities
14 Jan 2026 STEP3-VL-10B technical report 82.2 IFEval, SeRe (Table 2) Capabilities
11 Jan 2026 Solar Open 100B technical report 88 IFEval (Table 6) Capabilities
5 Jan 2026 K-EXAONE technical report 89.7 IFEval, prompt-strict (Table 3, Reasoning mode). Non-reasoning 85.5 (Table 4) Capabilities
15 Dec 2025 Olmo 3 32B Think technical report 89 IFEval, Final Think 3.0 column (Tables 14 and 24) Capabilities
15 Dec 2025 Olmo 3.1 32B Think technical report 93.8 IFEval (Tables 1, 14 and 24) Capabilities
28 Nov 2025 LFM2-8B-A1B technical report 77.6 IFEval, LFM2-8B-A1B (Table 7) Capabilities
28 Nov 2025 LFM2-1.2B technical report 74.9 IFEval, LFM2-1.2B (Table 6) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 88.2 IFEval (Table 6) Capabilities
8 Nov 2025 Falcon-Arabic-7B technical report 57.6 IFEval 0-shot, English (Table 8) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 78.2 IFEval_prompt_strict, reasoning on, SFT stage 4 (Table 6). Instruction strict 84.7 Capabilities
31 Oct 2025 LongCat-Flash-Omni technical report 82.4 IFEval (acc) (Table 13) Capabilities
21 Oct 2025 Ring-1T technical report 85.2 IFEval (Table 1) Capabilities
8 Oct 2025 OLMo 2 1B Instruct technical report 70.1 IFE, IFEval (Table 7, Table 23) Capabilities
23 Sep 2025 LongCat-Flash-Thinking technical report 86.9 IFEval(strict prompt) (Table 2) Capabilities
22 Sep 2025 Qwen3-Omni-30B-A3B technical report 81 IFEval, Instruct (Table 4). Thinking 85.1 (Table 5) Capabilities
17 Sep 2025 Apertus 70B technical report 75.2 IFEval, prompt-level loose (Table 19) Capabilities
2 Sep 2025 Baichuan-M2-32B technical report 86 IFEval (Table 1) Capabilities
1 Sep 2025 LongCat-Flash-Chat technical report 89.7 IFEval (acc) (Table 3) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 83.7 IFEval (Table 2) Capabilities
25 Aug 2025 Hermes 4 405B technical report 81.5 IFEval (Loose), reasoning mode (Table 3). Non-reasoning 84.9 Capabilities
21 Aug 2025 Intern-S1 technical report 86.7 IFEval (Table 2) Capabilities
8 Aug 2025 GLM-4.5 technical report 86.1 IFEval (Prompt Strict) (Table 6) Capabilities
8 Aug 2025 GLM-4.5-Air technical report 86.3 IFEval (Prompt Strict) (Table 6) Capabilities
30 Jul 2025 Falcon-H1-34B technical report 89.4 IFEval, instruction and prompt average accuracy (Table 24) Capabilities
21 Jul 2025 Kimi K2 technical report 89.8 IFEval (Prompt Strict) (Table 3) Capabilities
17 Jul 2025 Apple Foundation Model Server (2025) technical report 89.1 IFEval (instruct), AFM Server at 16 bits (Table 3). AFM Server Opt at 3.6 bits per weight 90.2 Capabilities
15 Jul 2025 EXAONE 4.0 32B technical report 83.7 IFEval (Table 3, Reasoning). Non-reasoning 84.8 (Table 4) Capabilities
29 Jun 2025 ERNIE-4.5-300B-A47B technical report 88 IFEval (Table 5) Capabilities
26 Jun 2025 Hunyuan-A13B-Instruct technical report 84.7 IF-Eval, slow-thinking (Table 3). Fast-thinking 84.4 (Table 4) Capabilities
14 May 2025 Qwen3-8B technical report 85 IFEval strict prompt, thinking mode (Table 17). Non-thinking 83.0 (Table 18) Capabilities
14 May 2025 Qwen3-4B technical report 81.9 IFEval strict prompt, thinking mode (Table 17). Non-thinking 81.2 (Table 18) Capabilities
14 May 2025 Qwen3-32B technical report 85 IFEval strict prompt, thinking mode (Table 13). Non-thinking 83.2 (Table 14) Capabilities
14 May 2025 Qwen3-30B-A3B technical report 86.5 IFEval strict prompt, thinking mode (Table 15). Non-thinking 83.7 (Table 16) Capabilities
14 May 2025 Qwen3-235B-A22B technical report 83.4 IFEval strict prompt, thinking mode (Table 11). Non-thinking 83.2 (Table 12) Capabilities
14 May 2025 Qwen3-14B technical report 85.4 IFEval strict prompt, thinking mode (Table 15). Non-thinking 84.8 (Table 16) Capabilities
14 May 2025 Qwen3-1.7B technical report 72.5 IFEval strict prompt, thinking mode (Table 19). Non-thinking 68.2 (Table 20) Capabilities
13 May 2025 Aya Vision 32B technical report 78.5 IFEval, English only (Table 5) Capabilities
12 May 2025 MiMo-7B-RL technical report 61 IF-Eval (Prompt Strict), average of 8 runs (Table 4) Capabilities
12 May 2025 INTELLECT-2 technical report 81.5 IFEval, evalchemy defaults, target length 10,000 tokens (Table 1) Capabilities
2 May 2025 Llama-3.3-Nemotron-Super-49B-v1 technical report 89.2 IFEval (Strict-Instruction), reasoning on, reasoning off 89.0 (Table 4) Capabilities
2 May 2025 Llama-3.1-Nemotron-Ultra-253B-v1 technical report 88.9 IFEval (Strict-Instruction), reasoning on, reasoning off 89.5 (Table 5) Capabilities
30 Apr 2025 Phi-4-reasoning technical report 83.4 IFEval Strict (Table 2) Capabilities
30 Apr 2025 Phi-4-Reasoning-plus technical report 84.9 IFEval Strict (Table 2) Capabilities
30 Apr 2025 Amazon Nova Premier technical report 91.5 IFEval, instruction-level loose accuracy, +/-2.4 (Table 1) Capabilities
16 Apr 2025 BitNet b1.58 2B4T technical report 53.5 IFEval (0-shot; Instruct-Strict) (Table 1, Table 2) Capabilities
10 Apr 2025 Seed-Thinking-v1.5 technical report 87.4 IFEval (Table 2) Capabilities
1 Apr 2025 Cohere Command A technical report 90.9 IFEval, mean of prompt-level and instruction-level strict accuracy (Table 3) Capabilities
25 Mar 2025 Gemma 3 4B technical report 90.2 IFEval, IT, 0-shot (Table 18) Capabilities
25 Mar 2025 Gemma 3 27B technical report 90.4 IFEval, IT, 0-shot (Table 18) Capabilities
25 Mar 2025 Gemma 3 1B technical report 80.2 IFEval, IT, 0-shot (Table 18) Capabilities
25 Mar 2025 Gemma 3 12B technical report 88.9 IFEval, IT, 0-shot (Table 18) Capabilities
3 Mar 2025 Phi-4 Mini technical report 70.1 IFEval (0-shot) (Table 7) Capabilities
19 Feb 2025 Qwen2.5-VL-72B technical report 86.3 IFEval (Table 4) Capabilities
26 Jan 2025 Qwen2.5-14B-Instruct-1M technical report 84.3 IFEval, strict prompt-level accuracy (Table 6) Capabilities
20 Jan 2025 Kimi k1.5 technical report 87.2 IF-Eval (Prompt Strict), short-CoT (Table 3) Capabilities
20 Jan 2025 DeepSeek-R1 technical report 83.3 IF-Eval (Prompt Strict), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
14 Jan 2025 MiniMax-Text-01 technical report 89.1 IFEval (avg) (Table 8) Capabilities
31 Dec 2024 OLMo 2 32B Instruct technical report 85.6 IFEval, prompt loose (Tables 7 and 16) Capabilities
26 Dec 2024 DeepSeek-V3 technical report 86.1 IF-Eval (Prompt Strict), DeepSeek-V3 (Table 6) Capabilities
19 Dec 2024 Qwen2.5-7B technical report 71.2 IFEval, Qwen2.5-7B-Instruct (Table 8) Capabilities
19 Dec 2024 Qwen2.5-72B technical report 84.1 IFEval, Qwen2.5-72B-Instruct (Table 6) Capabilities
19 Dec 2024 Qwen2.5-32B technical report 79.5 IFEval, Qwen2.5-32B-Instruct (Table 7) Capabilities
19 Dec 2024 Qwen2.5-14B technical report 81 IFEval, Qwen2.5-14B-Instruct (Table 7) Capabilities
19 Dec 2024 Qwen-Turbo technical report 76.3 IFEval, Qwen2.5-Turbo (Table 7) Capabilities
12 Dec 2024 Phi-4 technical report 63 IFEval (Table 1) Capabilities
6 Dec 2024 Llama 3.3 70B Instruct model card 92.1 IFEval (Instruction tuned models table) Capabilities
23 Nov 2024 Llama 3.1-8B technical report 80.4 IFEval, post-trained (Table 2) Capabilities
23 Nov 2024 Llama 3.1-70B technical report 87.5 IFEval, post-trained (Table 2) Capabilities
23 Nov 2024 Llama 3.1-405B technical report 88.6 IFEval, post-trained (Table 2) Capabilities
22 Nov 2024 Tulu 3 405B technical report 86 IFEval (loose prompt), RLVR (Table 4) Capabilities
22 Oct 2024 Claude 3.5 Sonnet (October 2024) model card 90.2 IFEval (Table 7) Capabilities
22 Oct 2024 Claude 3.5 Haiku model card 85.9 IFEval (Table 8) Capabilities
23 Jul 2024 Llama 3.1-8B model card 80.4 IFEval, instruction tuned (Instruction tuned models table) Capabilities
23 Jul 2024 Llama 3.1-70B model card 87.5 IFEval, instruction tuned (Instruction tuned models table) Capabilities
23 Jul 2024 Llama 3.1-405B model card 88.6 IFEval, instruction tuned (Instruction tuned models table) Capabilities
15 Jul 2024 Qwen2-72B technical report 77.6 IFEval strict-prompt, Qwen2-72B-Instruct (Table 6) Capabilities
7 May 2024 DeepSeek-V2 (MoE-236B, May 2024) technical report 63.8 IFEval 0-shot, DeepSeek-V2 Chat (RL) (Table 3) Capabilities
5 Jan 2024 DeepSeek LLM 67B technical report 55.5 IFEval prompt-level loose, held-out (Table 9) Capabilities

Versions

Replaced by IFBench

More in Instructions and chat

All 76 in Instructions and chat
  • IFBench In use 58 new out-of-domain verifiable output constraints. Scored by programmatic constraint checks (strict or loose). Nemotron-Cascade 2 30B-A3B 82.9%
  • MultiChallenge In use Multi-turn conversations testing instruction retention, inference memory, versioned editing and self-coherence. Graded by LLM judge with rubrics. GPT-6.1 Sol 82%
  • Multi-IF In use Models follow verifiable instructions over three-turn conversations in eight languages, extending IFEval. Scored by instruction accuracy at each turn. LFM3-1.7B 89.1%
  • AdvancedIF In use Models respond to over 1,600 prompts with complex, multi-turn, and system-level instructions. A judge model checks each response against expert-written rubrics. MAI-Thinking-1 85%
  • IFStruct In use Models generate JSON or YAML that must follow a requested schema and phrasing constraints, without constrained decoding. A response passes only if every structural constraint holds. LFM3-1.7B 93.7%
  • NV Chat Benchmark In use Models answer chat prompts from an internal NVIDIA set. The Nemotron 3.5 Super VL card labels the score an LMArena v3 proxy. Nemotron 3.5 Super VL 52.7