Benchmark Almanac

ALRAGE

Qwen3-32B 80.7% leads since Apr 2025 Only in system cards

Models answer Arabic questions from retrieved passages of 40 Arabic books. A Qwen2.5-72B judge scores each answer against a gold answer on a 0 to 10 scale.

Llama 3.1 70B drafted the questions and native speakers checked them in a community sprint. It is the only generative task in OALL v2.

Made by Open Arabic LLM Leaderboard team (2A2I, TII and community) with Argilla Released Feb 2025 Reading in many languages All tasks public Last in a system card Nov 2025 Last model added Nov 2025 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
Dataset Leaderboard huggingface.co
151
Entries on its own leaderboard, 39 models we track
1
System card reports it, from 1 lab

Qwen3-32B leads with 80.7% among models released by Aug 2025

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Aug 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

151 entries. Scroll sideways for the rest.

  1. 80.7 Qwen3-32B
  2. 80.6 inceptionai/jais-family-30b-16k
  3. 80.6 Gemma 2 2B IT
  4. 80.5 recoilme recoilme-gemma-2-9B-v0.2
  5. 80.5 silma-ai SILMA-Kashif-2B-Instruct-v1.0
  6. 80.5 Gemma 2 9B
  7. 80.5 inceptionai/jais-family-30b-8k
  8. 80.5 Gemma 2 27B
  9. 80.4 silma-ai SILMA-9B-Instruct-v1.0
  10. 80.4 Qwen3-30B-A3B-Instruct
  11. 80.4 recoilme recoilme-gemma-2-9B-v0.4
  12. 80.2 MaziyarPanahi calme-2.1-qwen2.5-72b
  13. 80.2 inceptionai/jais-family-13b
  14. 80.0 FreedomIntelligence AceGPT-13B
  15. 79.9 MaziyarPanahi calme-2.2-qwen2.5-72b
  16. 79.8 Sakalti oxyge1-33B
  17. 79.8 Cran-May tempmotacilla-cinerea-0308
  18. 79.8 FreedomIntelligence AceGPT-13B-chat
  19. 79.7 Qwen3-14B
  20. 79.6 Aya Expanse 32B
  21. 79.4 Daemontatox Cogito-R1
  22. 79.3 TIGER-Lab Qwen2.5-32B-Instruct-CFT
  23. 79.3 maldv Awqward2.5-32B-Instruct
  24. 79.2 Qwen2.5-32B
  25. 79.1 qwen3-4b-instruct-2507
  26. 79.0 huihui-ai Qwen2.5-32B-Instruct-abliterated
  27. 79.0 Applied-Innovation-Center Karnak
  28. 79.0 zetasepic Qwen2.5-32B-Instruct-abliterated-v2
  29. 78.9 rombodawg Rombos-LLM-V2.6-Qwen-14b
  30. 78.9 Qwen2.5-14B
  31. 78.8 Qwen2.5-72B
  32. 78.8 v000000 Qwen2.5-Lumen-14B
  33. 78.7 maldv Qwentile2.5-32B-Instruct
  34. 78.7 v000000 Qwen2.5-14B-Gutenberg-1e-Delta
  35. 78.7 tanliboy lambda-qwen2.5-32b-dpo-test
  36. 78.5 tanliboy lambda-qwen2.5-14b-dpo-test
  37. 78.4 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.2
  38. 78.2 Qwen3-8B
  39. 77.8 tunny Arabic_Qwen2.5_72B_instruct_finetune_0.1
  40. 77.7 huihui-ai Qwen2.5-7B-Instruct-abliterated-v2
  41. 77.7 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.1
  42. 77.6 rombodawg Rombos-LLM-V2.5-Qwen-72b
  43. 77.5 Sakalti Saka-14B
  44. 77.5 Lina-Z qwen_arabic_ft
  45. 77.4 Sakalti Ultiima-72B
  46. 77.4 Qwen2.5-7B
  47. 77.3 FreedomIntelligence AceGPT-v1.5-13B
  48. 77.3 Sakalti SakaMoe-3x14B-Instruct
  49. 77.2 airev-ai emirati-14b-v2
  50. 77.2 Phi-4
  51. 77.0 hammh0a Hala-9B
  52. 77.0 QCRI/Fanar-1-9B-Instruct
  53. 77.0 yellowtown 7B-v0.2
  54. 76.9 FreedomIntelligence AceGPT-v2-32B-Chat
  55. 76.8 ALLaM-AI/ALLaM-7B-Instruct-preview
  56. 76.7 amine-khelif BARS
  57. 76.7 huihui-ai Qwen2.5-7B-Instruct-abliterated
  58. 76.6 Navid-AI Yehia-7B-preview
  59. 76.6 Orion-zhen Qwen2.5-7B-Instruct-Uncensored
  60. 76.3 Qwen3-4B
  61. 76.1 upstage SOLAR-10.7B-v1.0
  62. 76.0 Applied-Innovation-Center AIC-1
  63. 75.9 CohereForAI/c4ai-command-r7b-arabic-02-2025
  64. 75.9 AXCXEPT EZO-Qwen2.5-32B-Instruct
  65. 75.8 Lina-Z arabic_llm_test
  66. 75.8 Aya Expanse 8B
  67. 75.8 Llama 3.3 70B Instruct
  68. 75.8 Cran-May T.E-8.1
  69. 75.8 Isaak-Carter Josiefied-Qwen2.5-7B-Instruct-abliterated-v2
  70. 75.0 FreedomIntelligence AceGPT-7B-chat
  71. 75.0 inceptionai/jais-family-30b-16k-chat
  72. 74.8 AbdulmalekDS qwen72b-ar-lora
  73. 74.7 JasperV13 Yehia-7B-DPO-Reasoning-preview
  74. 73.7 CohereForAI/aya-23-35B
  75. 73.4 Qwen2-72B
  76. 73.3 Gemma 3 27B
  77. 73.3 freewheelin free-evo-qwen72b-v0.8-re
  78. 73.3 JasperV13 Yehia-7B-Reasoning-preview
  79. 73.0 Mushari440 qwen-3-8B-sft-grpo
  80. 72.8 FreedomIntelligence AceGPT-v2-70B-Chat
  81. 72.7 01-ai/Yi-1.5-9B
  82. 72.6 inceptionai/jais-family-30b-8k-chat
  83. 72.3 inceptionai/jais-adapted-70b-chat
  84. 71.7 Qwen2.5-3B
  85. 71.5 01-ZeroOne SUHAIL-14B-preview
  86. 71.1 Qwen2-7B-Instruct
  87. 70.9 FreedomIntelligence AceGPT-v2-8B-Chat
  88. 70.7 arcee-ai Arcee-Spark
  89. 70.3 VAGOsolutions SauerkrautLM-Nemo-12b-Instruct
  90. 69.1 Phi-4 Mini
  91. 69.0 asas-ai AceGPT-v2-32B
  92. 69.0 FreedomIntelligence AceGPT-v2-32B
  93. 68.9 FreedomIntelligence AceGPT-v2-70B
  94. 68.4 inceptionai/jais-adapted-13b-chat
  95. 68.1 speakleash Bielik-11B-v2
  96. 67.9 SeaLLMs SeaLLM-7B-v2.5
  97. 67.9 Mushari440 qwen3-8B-SFT
  98. 67.0 Qwen1.5-32B
  99. 67.0 Ocelotr Qwen-8B-SFT
  100. 66.5 FreedomIntelligence AceGPT-v1.5-13B-Chat
  101. 66.3 riotu-lab ArabianGPT-01B
  102. 66.3 Qwen3-1.7B
  103. 64.6 TarjamaN Pronoia-14b-community
  104. 63.8 SeaLLMs SeaLLMs-v3-7B-Chat
  105. 63.6 inceptionai/jais-adapted-7b-chat
  106. 63.2 JasperV13 Yehia-7B-SFT-Reasoning-preview
  107. 62.9 Qwen1.5-14B
  108. 62.5 inceptionai/jais-adapted-13b
  109. 62.1 AIDC-AI Marco-LLM-AR-V4
  110. 61.8 Hagrass LLama3-3.2-instruct-trained
  111. 61.7 DeepSeek-R1-Distill-Qwen-32B
  112. 61.6 Qwen2.5-1.5B
  113. 60.7 QwQ-32B
  114. 60.2 tiiuae/Falcon3-7B-Base
  115. 59.9 Qwen1.5-7B
  116. 56.8 tiiuae/Falcon3-10B-Base
  117. 56.3 FreedomIntelligence AceGPT-7B
  118. 56.1 cognitivecomputations Dolphin3.0-R1-Mistral-24B
  119. 55.9 DeepSeek-R1-Distill-Qwen-14B
  120. 55.3 inceptionai/jais-adapted-70b
  121. 54.8 Qwen2-1.5B-Instruct
  122. 53.5 Qwen3-0.6B
  123. 53.3 Llama 3.2 3B
  124. 52.5 mobiuslabsgmbh DeepSeek-R1-ReDistill-Llama3-8B-v1.1
  125. 52.3 QCRI/Fanar-1-9B
  126. 51.1 AIDC-AI Marco-LLM-AR-V2
  127. 49.9 Llama 3.1-8B
  128. 48.9 AIDC-AI Marco-LLM-AR-V3
  129. 48.8 INSAIT-Institute BgGPT-7B-Instruct-v0.2
  130. 47.9 Qwen/Qwen1.5-4B
  131. 44.8 Naseej noon-7b
  132. 44.6 01-ai/Yi-1.5-34B-32K
  133. 43.1 Llama 3.2 1B
  134. 42.8 TIGER-Lab Qwen2.5-Math-7B-CFT
  135. 42.6 Qwen2.5-0.5B
  136. 42.3 Syed-Hasan-8503 Phi-3-mini-4K-instruct-cpo-simpo
  137. 41.9 Qwen/Qwen1.5-1.8B
  138. 41.4 inceptionai/jais-adapted-7b
  139. 39.7 Ocelotr Qwen3-8B-GAE
  140. 38.5 Qwen2-0.5B-Instruct
  141. 38.5 Sakalti Saka-1.5B
  142. 38.1 01-ai/Yi-1.5-9B-32K
  143. 34.4 Mushari440 Qwen3-8B-SFT-V2
  144. 33.8 AIDC-AI Marco-LLM-AR
  145. 33.5 mobiuslabsgmbh DeepSeek-R1-ReDistill-Qwen-7B-v1.1
  146. 33.4 Gemma 3 4B
  147. 30.8 stabilityai ar-stablelm-2-base
  148. 28.5 inceptionai/jais-family-13b-chat
  149. 25.3 Gemma 3 12B
  150. 20.8 Slim205 Barka-9b-it
  151. 15.8 Slim205 Barka-2b-it
  1. 1 Qwen3-32B 80.7%
  2. 2 inceptionai/jais-family-30b-16k 80.6%
  3. 3 Gemma 2 2B IT 80.6%
  4. 4 recoilme recoilme-gemma-2-9B-v0.2 80.5%
  5. 5 silma-ai SILMA-Kashif-2B-Instruct-v1.0 80.5%
  6. 6 Gemma 2 9B 80.5%
  7. 7 inceptionai/jais-family-30b-8k 80.5%
  8. 8 Gemma 2 27B base model 80.5%
  9. 9 silma-ai SILMA-9B-Instruct-v1.0 80.4%
  10. 10 Qwen3-30B-A3B-Instruct Jul 2025 80.4%
  11. 11 recoilme recoilme-gemma-2-9B-v0.4 80.4%
  12. 12 MaziyarPanahi calme-2.1-qwen2.5-72b 80.2%
  13. 13 inceptionai/jais-family-13b 80.2%
  14. 14 FreedomIntelligence AceGPT-13B 80%
  15. 15 MaziyarPanahi calme-2.2-qwen2.5-72b 79.9%
  16. 16 Sakalti oxyge1-33B 79.8%
  17. 17 Cran-May tempmotacilla-cinerea-0308 79.8%
  18. 18 FreedomIntelligence AceGPT-13B-chat 79.8%
  19. 19 Qwen3-14B 79.7%
  20. 20 Aya Expanse 32B 79.6%
  21. 21 Daemontatox Cogito-R1 79.4%
  22. 22 TIGER-Lab Qwen2.5-32B-Instruct-CFT 79.3%
  23. 23 maldv Awqward2.5-32B-Instruct 79.3%
  24. 24 Qwen2.5-32B 79.2%
  25. 25 qwen3-4b-instruct-2507 79.1%
  26. 26 huihui-ai Qwen2.5-32B-Instruct-abliterated 79%
  27. 27 Applied-Innovation-Center Karnak 79%
  28. 28 zetasepic Qwen2.5-32B-Instruct-abliterated-v2 79%
  29. 29 rombodawg Rombos-LLM-V2.6-Qwen-14b 78.9%
  30. 30 Qwen2.5-14B 78.9%
  31. 31 Qwen2.5-72B 78.8%
  32. 32 v000000 Qwen2.5-Lumen-14B 78.8%
  33. 33 maldv Qwentile2.5-32B-Instruct 78.7%
  34. 34 v000000 Qwen2.5-14B-Gutenberg-1e-Delta 78.7%
  35. 35 tanliboy lambda-qwen2.5-32b-dpo-test 78.7%
  36. 36 tanliboy lambda-qwen2.5-14b-dpo-test 78.5%
  37. 37 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.2 78.4%
  38. 38 Qwen3-8B 78.2%
  39. 39 tunny Arabic_Qwen2.5_72B_instruct_finetune_0.1 77.8%
  40. 40 huihui-ai Qwen2.5-7B-Instruct-abliterated-v2 77.7%
  41. 41 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.1 77.7%
  42. 42 rombodawg Rombos-LLM-V2.5-Qwen-72b 77.6%
  43. 43 Sakalti Saka-14B 77.5%
  44. 44 Lina-Z qwen_arabic_ft 77.5%
  45. 45 Sakalti Ultiima-72B 77.4%
  46. 46 Qwen2.5-7B 77.4%
  47. 47 FreedomIntelligence AceGPT-v1.5-13B 77.3%
  48. 48 Sakalti SakaMoe-3x14B-Instruct 77.3%
  49. 49 airev-ai emirati-14b-v2 77.2%
  50. 50 Phi-4 77.2%
  51. 51 hammh0a Hala-9B 77%
  52. 52 QCRI/Fanar-1-9B-Instruct 77%
  53. 53 yellowtown 7B-v0.2 77%
  54. 54 FreedomIntelligence AceGPT-v2-32B-Chat 76.9%
  55. 55 ALLaM-AI/ALLaM-7B-Instruct-preview 76.8%
  56. 56 amine-khelif BARS 76.7%
  57. 57 huihui-ai Qwen2.5-7B-Instruct-abliterated 76.7%
  58. 58 Navid-AI Yehia-7B-preview 76.6%
  59. 59 Orion-zhen Qwen2.5-7B-Instruct-Uncensored 76.6%
  60. 60 Qwen3-4B 76.3%
  61. 61 upstage SOLAR-10.7B-v1.0 76.1%
  62. 62 Applied-Innovation-Center AIC-1 76%
  63. 63 CohereForAI/c4ai-command-r7b-arabic-02-2025 75.9%
  64. 64 AXCXEPT EZO-Qwen2.5-32B-Instruct 75.9%
  65. 65 Lina-Z arabic_llm_test 75.8%
  66. 66 Aya Expanse 8B 75.8%
  67. 67 Llama 3.3 70B Instruct 75.8%
  68. 68 Cran-May T.E-8.1 75.8%
  69. 69 Isaak-Carter Josiefied-Qwen2.5-7B-Instruct-abliterated-v2 75.8%
  70. 70 FreedomIntelligence AceGPT-7B-chat 75%
  71. 71 inceptionai/jais-family-30b-16k-chat 75%
  72. 72 AbdulmalekDS qwen72b-ar-lora 74.8%
  73. 73 JasperV13 Yehia-7B-DPO-Reasoning-preview 74.7%
  74. 74 CohereForAI/aya-23-35B 73.7%
  75. 75 Qwen2-72B base model 73.4%
  76. 76 Gemma 3 27B 73.3%
  77. 77 freewheelin free-evo-qwen72b-v0.8-re 73.3%
  78. 78 JasperV13 Yehia-7B-Reasoning-preview 73.3%
  79. 79 Mushari440 qwen-3-8B-sft-grpo 73%
  80. 80 FreedomIntelligence AceGPT-v2-70B-Chat 72.8%
  81. 81 01-ai/Yi-1.5-9B base model 72.7%
  82. 82 inceptionai/jais-family-30b-8k-chat 72.6%
  83. 83 inceptionai/jais-adapted-70b-chat 72.3%
  84. 84 Qwen2.5-3B 71.7%
  85. 85 01-ZeroOne SUHAIL-14B-preview 71.5%
  86. 86 Qwen2-7B-Instruct 71.1%
  87. 87 FreedomIntelligence AceGPT-v2-8B-Chat 70.9%
  88. 88 arcee-ai Arcee-Spark 70.7%
  89. 89 VAGOsolutions SauerkrautLM-Nemo-12b-Instruct 70.3%
  90. 90 Phi-4 Mini 69.1%
  91. 91 asas-ai AceGPT-v2-32B 69%
  92. 92 FreedomIntelligence AceGPT-v2-32B 69%
  93. 93 FreedomIntelligence AceGPT-v2-70B 68.9%
  94. 94 inceptionai/jais-adapted-13b-chat 68.4%
  95. 95 speakleash Bielik-11B-v2 68.1%
  96. 96 SeaLLMs SeaLLM-7B-v2.5 67.9%
  97. 97 Mushari440 qwen3-8B-SFT 67.9%
  98. 98 Qwen1.5-32B base model 67%
  99. 99 Ocelotr Qwen-8B-SFT 67%
  100. 100 FreedomIntelligence AceGPT-v1.5-13B-Chat 66.5%
  101. 101 riotu-lab ArabianGPT-01B 66.3%
  102. 102 Qwen3-1.7B base model 66.3%
  103. 103 TarjamaN Pronoia-14b-community 64.6%
  104. 104 SeaLLMs SeaLLMs-v3-7B-Chat 63.8%
  105. 105 inceptionai/jais-adapted-7b-chat 63.6%
  106. 106 JasperV13 Yehia-7B-SFT-Reasoning-preview 63.2%
  107. 107 Qwen1.5-14B base model 62.9%
  108. 108 inceptionai/jais-adapted-13b 62.5%
  109. 109 AIDC-AI Marco-LLM-AR-V4 62.1%
  110. 110 Hagrass LLama3-3.2-instruct-trained 61.8%
  111. 111 DeepSeek-R1-Distill-Qwen-32B 61.7%
  112. 112 Qwen2.5-1.5B 61.6%
  113. 113 QwQ-32B 60.7%
  114. 114 tiiuae/Falcon3-7B-Base base model 60.2%
  115. 115 Qwen1.5-7B base model 59.9%
  116. 116 tiiuae/Falcon3-10B-Base base model 56.8%
  117. 117 FreedomIntelligence AceGPT-7B 56.3%
  118. 118 cognitivecomputations Dolphin3.0-R1-Mistral-24B 56.1%
  119. 119 DeepSeek-R1-Distill-Qwen-14B 55.9%
  120. 120 inceptionai/jais-adapted-70b 55.3%
  121. 121 Qwen2-1.5B-Instruct 54.8%
  122. 122 Qwen3-0.6B 53.5%
  123. 123 Llama 3.2 3B 53.3%
  124. 124 mobiuslabsgmbh DeepSeek-R1-ReDistill-Llama3-8B-v1.1 52.5%
  125. 125 QCRI/Fanar-1-9B 52.3%
  126. 126 AIDC-AI Marco-LLM-AR-V2 51.1%
  127. 127 Llama 3.1-8B 49.9%
  128. 128 AIDC-AI Marco-LLM-AR-V3 48.9%
  129. 129 INSAIT-Institute BgGPT-7B-Instruct-v0.2 48.8%
  130. 130 Qwen/Qwen1.5-4B base model 47.9%
  131. 131 Naseej noon-7b 44.8%
  132. 132 01-ai/Yi-1.5-34B-32K base model 44.6%
  133. 133 Llama 3.2 1B base model 43.1%
  134. 134 TIGER-Lab Qwen2.5-Math-7B-CFT 42.8%
  135. 135 Qwen2.5-0.5B 42.6%
  136. 136 Syed-Hasan-8503 Phi-3-mini-4K-instruct-cpo-simpo 42.3%
  137. 137 Qwen/Qwen1.5-1.8B base model 41.9%
  138. 138 inceptionai/jais-adapted-7b 41.4%
  139. 139 Ocelotr Qwen3-8B-GAE 39.7%
  140. 140 Qwen2-0.5B-Instruct 38.5%
  141. 141 Sakalti Saka-1.5B 38.5%
  142. 142 01-ai/Yi-1.5-9B-32K base model 38.1%
  143. 143 Mushari440 Qwen3-8B-SFT-V2 34.4%
  144. 144 AIDC-AI Marco-LLM-AR 33.8%
  145. 145 mobiuslabsgmbh DeepSeek-R1-ReDistill-Qwen-7B-v1.1 33.5%
  146. 146 Gemma 3 4B base model 33.4%
  147. 147 stabilityai ar-stablelm-2-base 30.8%
  148. 148 inceptionai/jais-family-13b-chat 28.5%
  149. 149 Gemma 3 12B base model 25.3%
  150. 150 Slim205 Barka-9b-it 20.8%
  151. 151 Slim205 Barka-2b-it 15.8%
#EntryScoreSettingDateSource
1 Qwen3-32B 80.7% Apr 2025 Official leaderboard
2 inceptionai/jais-family-30b-16k 80.6% Jan 2025 Official leaderboard
3 Gemma 2 2B IT 80.6% Jul 2024 Official leaderboard
4 recoilme recoilme-gemma-2-9B-v0.2 80.5% Jan 2025 Official leaderboard
5 silma-ai SILMA-Kashif-2B-Instruct-v1.0 80.5% Feb 2025 Official leaderboard
6 Gemma 2 9B 80.5% Jun 2024 Official leaderboard
7 inceptionai/jais-family-30b-8k 80.5% Jan 2025 Official leaderboard
8 Gemma 2 27B 80.5% base model Jun 2024 Official leaderboard
9 silma-ai SILMA-9B-Instruct-v1.0 80.4% Jan 2025 Official leaderboard
10 Qwen3-30B-A3B-Instruct Jul 2025 80.4% Jul 2025 Official leaderboard
11 recoilme recoilme-gemma-2-9B-v0.4 80.4% Jan 2025 Official leaderboard
12 MaziyarPanahi calme-2.1-qwen2.5-72b 80.2% Jan 2025 Official leaderboard
13 inceptionai/jais-family-13b 80.2% Jan 2025 Official leaderboard
14 FreedomIntelligence AceGPT-13B 80% Jan 2025 Official leaderboard
15 MaziyarPanahi calme-2.2-qwen2.5-72b 79.9% Jan 2025 Official leaderboard
16 Sakalti oxyge1-33B 79.8% Mar 2025 Official leaderboard
17 Cran-May tempmotacilla-cinerea-0308 79.8% Mar 2025 Official leaderboard
18 FreedomIntelligence AceGPT-13B-chat 79.8% Jan 2025 Official leaderboard
19 Qwen3-14B 79.7% Apr 2025 Official leaderboard
20 Aya Expanse 32B 79.6% Oct 2024 Official leaderboard
21 Daemontatox Cogito-R1 79.4% Feb 2025 Official leaderboard
22 TIGER-Lab Qwen2.5-32B-Instruct-CFT 79.3% Mar 2025 Official leaderboard
23 maldv Awqward2.5-32B-Instruct 79.3% Feb 2025 Official leaderboard
24 Qwen2.5-32B 79.2% Sep 2024 Official leaderboard
25 qwen3-4b-instruct-2507 79.1% Aug 2025 Official leaderboard
26 huihui-ai Qwen2.5-32B-Instruct-abliterated 79% Jan 2025 Official leaderboard
27 Applied-Innovation-Center Karnak 79% Feb 2026 Official leaderboard
28 zetasepic Qwen2.5-32B-Instruct-abliterated-v2 79% Mar 2025 Official leaderboard
29 rombodawg Rombos-LLM-V2.6-Qwen-14b 78.9% Jan 2025 Official leaderboard
30 Qwen2.5-14B 78.9% Sep 2024 Official leaderboard
31 Qwen2.5-72B 78.8% Sep 2024 Official leaderboard
32 v000000 Qwen2.5-Lumen-14B 78.8% Jan 2025 Official leaderboard
33 maldv Qwentile2.5-32B-Instruct 78.7% Feb 2025 Official leaderboard
34 v000000 Qwen2.5-14B-Gutenberg-1e-Delta 78.7% Jan 2025 Official leaderboard
35 tanliboy lambda-qwen2.5-32b-dpo-test 78.7% Jan 2025 Official leaderboard
36 tanliboy lambda-qwen2.5-14b-dpo-test 78.5% Jan 2025 Official leaderboard
37 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.2 78.4% Sep 2025 Official leaderboard
38 Qwen3-8B 78.2% Apr 2025 Official leaderboard
39 tunny Arabic_Qwen2.5_72B_instruct_finetune_0.1 77.8% Dec 2025 Official leaderboard
40 huihui-ai Qwen2.5-7B-Instruct-abliterated-v2 77.7% Jan 2025 Official leaderboard
41 deep-analysis-research D2IL-Arabic-Qwen2.5-72B-Instruct-v0.1 77.7% Jul 2025 Official leaderboard
42 rombodawg Rombos-LLM-V2.5-Qwen-72b 77.6% Feb 2025 Official leaderboard
43 Sakalti Saka-14B 77.5% Feb 2025 Official leaderboard
44 Lina-Z qwen_arabic_ft 77.5% Jan 2026 Official leaderboard
45 Sakalti Ultiima-72B 77.4% Feb 2025 Official leaderboard
46 Qwen2.5-7B 77.4% Sep 2024 Official leaderboard
47 FreedomIntelligence AceGPT-v1.5-13B 77.3% Jan 2025 Official leaderboard
48 Sakalti SakaMoe-3x14B-Instruct 77.3% Feb 2025 Official leaderboard
49 airev-ai emirati-14b-v2 77.2% Jan 2025 Official leaderboard
50 Phi-4 77.2% Dec 2024 Official leaderboard
51 hammh0a Hala-9B 77% Feb 2026 Official leaderboard
52 QCRI/Fanar-1-9B-Instruct 77% Jun 2025 Official leaderboard
53 yellowtown 7B-v0.2 77% Jan 2025 Official leaderboard
54 FreedomIntelligence AceGPT-v2-32B-Chat 76.9% Jan 2025 Official leaderboard
55 ALLaM-AI/ALLaM-7B-Instruct-preview 76.8% Feb 2025 Official leaderboard
56 amine-khelif BARS 76.7% Sep 2025 Official leaderboard
57 huihui-ai Qwen2.5-7B-Instruct-abliterated 76.7% Jan 2025 Official leaderboard
58 Navid-AI Yehia-7B-preview 76.6% Mar 2025 Official leaderboard
59 Orion-zhen Qwen2.5-7B-Instruct-Uncensored 76.6% Jan 2025 Official leaderboard
60 Qwen3-4B 76.3% Apr 2025 Official leaderboard
61 upstage SOLAR-10.7B-v1.0 76.1% Jan 2025 Official leaderboard
62 Applied-Innovation-Center AIC-1 76% Sep 2025 Official leaderboard
63 CohereForAI/c4ai-command-r7b-arabic-02-2025 75.9% Feb 2025 Official leaderboard
64 AXCXEPT EZO-Qwen2.5-32B-Instruct 75.9% Jan 2025 Official leaderboard
65 Lina-Z arabic_llm_test 75.8% Nov 2025 Official leaderboard
66 Aya Expanse 8B 75.8% Oct 2024 Official leaderboard
67 Llama 3.3 70B Instruct 75.8% Dec 2024 Official leaderboard
68 Cran-May T.E-8.1 75.8% Jan 2025 Official leaderboard
69 Isaak-Carter Josiefied-Qwen2.5-7B-Instruct-abliterated-v2 75.8% Jan 2025 Official leaderboard
70 FreedomIntelligence AceGPT-7B-chat 75% Jan 2025 Official leaderboard
71 inceptionai/jais-family-30b-16k-chat 75% Jan 2025 Official leaderboard
72 AbdulmalekDS qwen72b-ar-lora 74.8% Sep 2025 Official leaderboard
73 JasperV13 Yehia-7B-DPO-Reasoning-preview 74.7% May 2025 Official leaderboard
74 CohereForAI/aya-23-35B 73.7% Jan 2025 Official leaderboard
75 Qwen2-72B 73.4% base model Jun 2024 Official leaderboard
76 Gemma 3 27B 73.3% Mar 2025 Official leaderboard
77 freewheelin free-evo-qwen72b-v0.8-re 73.3% Jan 2025 Official leaderboard
78 JasperV13 Yehia-7B-Reasoning-preview 73.3% May 2025 Official leaderboard
79 Mushari440 qwen-3-8B-sft-grpo 73% Feb 2026 Official leaderboard
80 FreedomIntelligence AceGPT-v2-70B-Chat 72.8% Jan 2025 Official leaderboard
81 01-ai/Yi-1.5-9B 72.7% base model Jan 2025 Official leaderboard
82 inceptionai/jais-family-30b-8k-chat 72.6% Jan 2025 Official leaderboard
83 inceptionai/jais-adapted-70b-chat 72.3% Jan 2025 Official leaderboard
84 Qwen2.5-3B 71.7% Sep 2024 Official leaderboard
85 01-ZeroOne SUHAIL-14B-preview 71.5% Jul 2025 Official leaderboard
86 Qwen2-7B-Instruct 71.1% Jun 2024 Official leaderboard
87 FreedomIntelligence AceGPT-v2-8B-Chat 70.9% Jan 2025 Official leaderboard
88 arcee-ai Arcee-Spark 70.7% Jan 2025 Official leaderboard
89 VAGOsolutions SauerkrautLM-Nemo-12b-Instruct 70.3% Jan 2025 Official leaderboard
90 Phi-4 Mini 69.1% Feb 2025 Official leaderboard
91 asas-ai AceGPT-v2-32B 69% Jan 2025 Official leaderboard
92 FreedomIntelligence AceGPT-v2-32B 69% Jan 2025 Official leaderboard
93 FreedomIntelligence AceGPT-v2-70B 68.9% Jan 2025 Official leaderboard
94 inceptionai/jais-adapted-13b-chat 68.4% Jan 2025 Official leaderboard
95 speakleash Bielik-11B-v2 68.1% Jan 2025 Official leaderboard
96 SeaLLMs SeaLLM-7B-v2.5 67.9% Jan 2025 Official leaderboard
97 Mushari440 qwen3-8B-SFT 67.9% Feb 2026 Official leaderboard
98 Qwen1.5-32B 67% base model Feb 2024 Official leaderboard
99 Ocelotr Qwen-8B-SFT 67% Dec 2025 Official leaderboard
100 FreedomIntelligence AceGPT-v1.5-13B-Chat 66.5% Jan 2025 Official leaderboard
101 riotu-lab ArabianGPT-01B 66.3% Oct 2025 Official leaderboard
102 Qwen3-1.7B 66.3% base model Apr 2025 Official leaderboard
103 TarjamaN Pronoia-14b-community 64.6% Feb 2025 Official leaderboard
104 SeaLLMs SeaLLMs-v3-7B-Chat 63.8% Mar 2025 Official leaderboard
105 inceptionai/jais-adapted-7b-chat 63.6% Jan 2025 Official leaderboard
106 JasperV13 Yehia-7B-SFT-Reasoning-preview 63.2% May 2025 Official leaderboard
107 Qwen1.5-14B 62.9% base model Feb 2024 Official leaderboard
108 inceptionai/jais-adapted-13b 62.5% Jan 2025 Official leaderboard
109 AIDC-AI Marco-LLM-AR-V4 62.1% Mar 2025 Official leaderboard
110 Hagrass LLama3-3.2-instruct-trained 61.8% Aug 2025 Official leaderboard
111 DeepSeek-R1-Distill-Qwen-32B 61.7% Jan 2025 Official leaderboard
112 Qwen2.5-1.5B 61.6% Sep 2024 Official leaderboard
113 QwQ-32B 60.7% Mar 2025 Official leaderboard
114 tiiuae/Falcon3-7B-Base 60.2% base model Feb 2025 Official leaderboard
115 Qwen1.5-7B 59.9% base model Feb 2024 Official leaderboard
116 tiiuae/Falcon3-10B-Base 56.8% base model Feb 2025 Official leaderboard
117 FreedomIntelligence AceGPT-7B 56.3% Jan 2025 Official leaderboard
118 cognitivecomputations Dolphin3.0-R1-Mistral-24B 56.1% Feb 2025 Official leaderboard
119 DeepSeek-R1-Distill-Qwen-14B 55.9% Jan 2025 Official leaderboard
120 inceptionai/jais-adapted-70b 55.3% Jan 2025 Official leaderboard
121 Qwen2-1.5B-Instruct 54.8% Jun 2024 Official leaderboard
122 Qwen3-0.6B 53.5% Apr 2025 Official leaderboard
123 Llama 3.2 3B 53.3% Sep 2024 Official leaderboard
124 mobiuslabsgmbh DeepSeek-R1-ReDistill-Llama3-8B-v1.1 52.5% Feb 2025 Official leaderboard
125 QCRI/Fanar-1-9B 52.3% Jun 2025 Official leaderboard
126 AIDC-AI Marco-LLM-AR-V2 51.1% Mar 2025 Official leaderboard
127 Llama 3.1-8B 49.9% Jul 2024 Official leaderboard
128 AIDC-AI Marco-LLM-AR-V3 48.9% Mar 2025 Official leaderboard
129 INSAIT-Institute BgGPT-7B-Instruct-v0.2 48.8% Jan 2025 Official leaderboard
130 Qwen/Qwen1.5-4B 47.9% base model Jan 2025 Official leaderboard
131 Naseej noon-7b 44.8% Mar 2025 Official leaderboard
132 01-ai/Yi-1.5-34B-32K 44.6% base model Jan 2025 Official leaderboard
133 Llama 3.2 1B 43.1% base model Sep 2024 Official leaderboard
134 TIGER-Lab Qwen2.5-Math-7B-CFT 42.8% Mar 2025 Official leaderboard
135 Qwen2.5-0.5B 42.6% Sep 2024 Official leaderboard
136 Syed-Hasan-8503 Phi-3-mini-4K-instruct-cpo-simpo 42.3% Jan 2025 Official leaderboard
137 Qwen/Qwen1.5-1.8B 41.9% base model Jan 2025 Official leaderboard
138 inceptionai/jais-adapted-7b 41.4% Jan 2025 Official leaderboard
139 Ocelotr Qwen3-8B-GAE 39.7% Jan 2026 Official leaderboard
140 Qwen2-0.5B-Instruct 38.5% Jun 2024 Official leaderboard
141 Sakalti Saka-1.5B 38.5% Feb 2025 Official leaderboard
142 01-ai/Yi-1.5-9B-32K 38.1% base model Jan 2025 Official leaderboard
143 Mushari440 Qwen3-8B-SFT-V2 34.4% Mar 2026 Official leaderboard
144 AIDC-AI Marco-LLM-AR 33.8% Feb 2025 Official leaderboard
145 mobiuslabsgmbh DeepSeek-R1-ReDistill-Qwen-7B-v1.1 33.5% Feb 2025 Official leaderboard
146 Gemma 3 4B 33.4% base model Mar 2025 Official leaderboard
147 stabilityai ar-stablelm-2-base 30.8% Jun 2025 Official leaderboard
148 inceptionai/jais-family-13b-chat 28.5% Jan 2025 Official leaderboard
149 Gemma 3 12B 25.3% base model Mar 2025 Official leaderboard
150 Slim205 Barka-9b-it 20.8% Jun 2025 Official leaderboard
151 Slim205 Barka-2b-it 15.8% Jun 2025 Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Gemma 2 27B
base model
80.5%
Qwen3-14B
base model
79.7%
Qwen2.5-32B
base model
79.2%
Qwen2.5-14B
base model
78.9%
Qwen2.5-72B
base model
78.8%
Qwen3-8B
base model
78.2%
Qwen2.5-7B
base model
77.4%
Qwen3-4B
base model
76.3%
Gemma 3 27B
base model
73.3%
Qwen2.5-3B
base model
71.7%
Qwen2.5-1.5B
base model
61.6%
Llama 3.1-8B
base model
49.9%
ModelEffortHarnessScoreCost per taskOutput tokens
Qwen3-32B 80.7%
Gemma 2 2B IT 80.6%
Gemma 2 9B 80.5%
Gemma 2 27B base model 80.5%
Qwen3-30B-A3B-Instruct Jul 2025 80.4%
Gemma 2 27B 79.9%
Qwen3-14B 79.7%
Aya Expanse 32B 79.6%
Qwen2.5-32B 79.2%
qwen3-4b-instruct-2507 79.1%
Qwen2.5-14B 78.9%
Qwen2.5-72B 78.8%
Qwen3-8B 78.2%
Qwen2.5-72B base model 77.4%
Qwen2.5-7B 77.4%
Phi-4 77.2%
Qwen2.5-32B base model 76.7%
Qwen3-4B 76.3%
Aya Expanse 8B 75.8%
Llama 3.3 70B Instruct 75.8%
Qwen3-8B base model 74.1%
Qwen2-72B base model 73.4%
Gemma 3 27B 73.3%
Qwen3-14B base model 73.2%
Qwen2.5-14B base model 72.7%
Qwen2.5-3B 71.7%
Qwen2-7B-Instruct 71.1%
Qwen3-4B base model 70.9%
Phi-4 Mini 69.1%
Qwen1.5-32B base model 67%
Qwen3-1.7B base model 66.3%
Qwen2.5-7B base model 64.8%
Qwen1.5-14B base model 62.9%
Qwen2.5-3B base model 61.9%
DeepSeek-R1-Distill-Qwen-32B 61.7%
Qwen2.5-1.5B 61.6%
QwQ-32B 60.7%
Qwen1.5-7B base model 59.9%
DeepSeek-R1-Distill-Qwen-14B 55.9%
Qwen2-1.5B-Instruct 54.8%
Qwen3-0.6B 53.5%
Llama 3.2 3B 53.3%
Llama 3.1-8B 49.9%
Llama 3.1-8B base model 47.1%
Llama 3.2 1B base model 43.1%
Qwen2.5-0.5B 42.6%
Qwen2-0.5B-Instruct 38.5%
Gemma 3 4B base model 33.4%
Gemma 3 12B base model 25.3%
Qwen2.5-0.5B base model 25%
Gemma 3 27B base model 24.2%
Qwen2.5-1.5B base model 3.46%

2 models have held number one since its release in Feb 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Qwen
Own leaderboard Not tested since Aug 2025
Qwen3-32B 80.7%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3-32B Gemma 2 2B IT Qwen2-72B Qwen1.5-32B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3-32B Qwen2-72B

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
8 Nov 2025 Falcon-Arabic-7B technical report 72.3 ALRAGE, judged by Qwen2.5-72B-Instruct (Table 6) Capabilities

More in Multilingual

All 115 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Inkling 88.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%