Benchmark Almanac

ThaiExam

Claude 3.5 Sonnet 75.1% leads since Sep 2024 Not in model cards

Models answer multiple-choice questions from standardized examinations in Thailand, such as ONET, IC, TGAT, TPAT-1 and A-Level.

HELM releases v1.0.0 (2024-09-04) to v1.2.0 (2025-06-18).

Made by SCB 10X + Stanford CRFM Released Sep 2024 Knowledge in many languages All tasks public Last model added Aug 2024 Scores refreshed 2 Nov 2026
Level
School Students, from grade school to high school
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
stanford-crfm/helm 2,961 stars Leaderboard crfm.stanford.edu
42
Entries on its own leaderboard, 24 models we track

Claude 3.5 Sonnet leads with 75.1% among models released by Aug 2024

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Aug 2024, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

42 entries. Scroll sideways for the rest.

  1. 75.1 Claude 3.5 Sonnet
  2. 68.6 Claude 3 Opus
  3. 68.2 Llama 3.1-405B
  4. 64.7 Gemini 1.5 Pro
  5. 63.8 Qwen2-72B
  6. 63.3 GPT-4o
  7. 62.3 Llama 3.1-70B
  8. 61.7 llama-3-typhoon-v1.5x-70b-instruct
  9. 59.8 Gemini 1.5 Flash
  10. 59.5 typhoon-v1.5-72b-instruct
  11. 59.3 OpenThaiLLM-Prebuilt-7B
  12. 57.9 Claude 3 Sonnet
  13. 57.5 GPT-4 Turbo
  14. 56.8 Llama 3-70B
  15. 55.7 GPT-4o mini
  16. 54.0 Claude 3 Haiku
  17. 53.9 Pathumma-llm-text-1.0.0
  18. 53.8 Qwen1.5-72B
  19. 53.0 llama3-8b-cpt-sea-lionv2.1-instruct
  20. 51.8 sambalingo-thai-chat-70b
  21. 49.3 llama-3-typhoon-v1.5x-8b-instruct
  22. 48.5 sailor-14b-chat
  23. 47.9 typhoon-v1.5-8b-instruct
  24. 47.8 seallm-7b-v2.5
  25. 47.7 Gemini 1.0 Pro
  26. 45.4 llama3-8b-cpt-sea-lionv2-base
  27. 42.3 typhoon-7b
  28. 41.6 Qwen1.5-7B-Chat
  29. 39.3 sailor-7b-chat
  30. 39.2 GPT-3.5 Turbo
  31. 37.9 Command R+
  32. 37.8 openthaigpt-1.0.0-13b-chat
  33. 37.7 seallm-7b-v2
  34. 37.7 Mixtral 8x22B
  35. 37.7 Command R (03-2024)
  36. 33.6 sambalingo-thai-chat
  37. 32.0 Mistral 7B v0.3
  38. 31.1 openthaigpt-1.0.0-7b-chat
  39. 30.0 Mixtral 8x7B
  40. 24.0 Llama 3.1-8B
  41. 20.2 sea-lion-7b-instruct
  42. 19.8 Llama 3-8B
  1. 1 Claude 3.5 Sonnet 75.1%
  2. 2 Claude 3 Opus 68.6%
  3. 3 Llama 3.1-405B 68.2%
  4. 4 Gemini 1.5 Pro May 2024 64.7%
  5. 5 Qwen2-72B 63.8%
  6. 6 GPT-4o May 2024 63.3%
  7. 7 Llama 3.1-70B 62.3%
  8. 8 llama-3-typhoon-v1.5x-70b-instruct 61.7%
  9. 9 Gemini 1.5 Flash May 2024 59.8%
  10. 10 typhoon-v1.5-72b-instruct 59.5%
  11. 11 OpenThaiLLM-Prebuilt-7B 59.3%
  12. 12 Claude 3 Sonnet 57.9%
  13. 13 GPT-4 Turbo Apr 2024 57.5%
  14. 14 Llama 3-70B 56.8%
  15. 15 GPT-4o mini 55.7%
  16. 16 Claude 3 Haiku 54%
  17. 17 Pathumma-llm-text-1.0.0 53.9%
  18. 18 Qwen1.5-72B 53.8%
  19. 19 llama3-8b-cpt-sea-lionv2.1-instruct 53%
  20. 20 sambalingo-thai-chat-70b 51.8%
  21. 21 llama-3-typhoon-v1.5x-8b-instruct 49.3%
  22. 22 sailor-14b-chat 48.5%
  23. 23 typhoon-v1.5-8b-instruct 47.9%
  24. 24 seallm-7b-v2.5 47.8%
  25. 25 Gemini 1.0 Pro 47.7%
  26. 26 llama3-8b-cpt-sea-lionv2-base 45.4%
  27. 27 typhoon-7b 42.3%
  28. 28 Qwen1.5-7B-Chat 41.6%
  29. 29 sailor-7b-chat 39.3%
  30. 30 GPT-3.5 Turbo Jan 2024 39.2%
  31. 31 Command R+ 37.9%
  32. 32 openthaigpt-1.0.0-13b-chat 37.8%
  33. 33 seallm-7b-v2 37.7%
  34. 34 Mixtral 8x22B 37.7%
  35. 35 Command R (03-2024) 37.7%
  36. 36 sambalingo-thai-chat 33.6%
  37. 37 Mistral 7B v0.3 32%
  38. 38 openthaigpt-1.0.0-7b-chat 31.1%
  39. 39 Mixtral 8x7B 30%
  40. 40 Llama 3.1-8B 24%
  41. 41 sea-lion-7b-instruct 20.2%
  42. 42 Llama 3-8B 19.8%
#EntryScoreSettingDateSource
1 Claude 3.5 Sonnet 75.1% Jun 2024 Official leaderboard
2 Claude 3 Opus 68.6% Feb 2024 Official leaderboard
3 Llama 3.1-405B 68.2% Jul 2024 Official leaderboard
4 Gemini 1.5 Pro May 2024 64.7% May 2024 Official leaderboard
5 Qwen2-72B 63.8% Jun 2024 Official leaderboard
6 GPT-4o May 2024 63.3% May 2024 Official leaderboard
7 Llama 3.1-70B 62.3% Jul 2024 Official leaderboard
8 llama-3-typhoon-v1.5x-70b-instruct 61.7% Official leaderboard
9 Gemini 1.5 Flash May 2024 59.8% May 2024 Official leaderboard
10 typhoon-v1.5-72b-instruct 59.5% Official leaderboard
11 OpenThaiLLM-Prebuilt-7B 59.3% Official leaderboard
12 Claude 3 Sonnet 57.9% Feb 2024 Official leaderboard
13 GPT-4 Turbo Apr 2024 57.5% Apr 2024 Official leaderboard
14 Llama 3-70B 56.8% Apr 2024 Official leaderboard
15 GPT-4o mini 55.7% Jul 2024 Official leaderboard
16 Claude 3 Haiku 54% Mar 2024 Official leaderboard
17 Pathumma-llm-text-1.0.0 53.9% Official leaderboard
18 Qwen1.5-72B 53.8% Feb 2024 Official leaderboard
19 llama3-8b-cpt-sea-lionv2.1-instruct 53% Official leaderboard
20 sambalingo-thai-chat-70b 51.8% Official leaderboard
21 llama-3-typhoon-v1.5x-8b-instruct 49.3% Official leaderboard
22 sailor-14b-chat 48.5% Official leaderboard
23 typhoon-v1.5-8b-instruct 47.9% Official leaderboard
24 seallm-7b-v2.5 47.8% Official leaderboard
25 Gemini 1.0 Pro 47.7% Dec 2023 Official leaderboard
26 llama3-8b-cpt-sea-lionv2-base 45.4% Official leaderboard
27 typhoon-7b 42.3% Official leaderboard
28 Qwen1.5-7B-Chat 41.6% Feb 2024 Official leaderboard
29 sailor-7b-chat 39.3% Official leaderboard
30 GPT-3.5 Turbo Jan 2024 39.2% Jan 2024 Official leaderboard
31 Command R+ 37.9% Aug 2024 Official leaderboard
32 openthaigpt-1.0.0-13b-chat 37.8% Official leaderboard
33 seallm-7b-v2 37.7% Official leaderboard
34 Mixtral 8x22B 37.7% Apr 2024 Official leaderboard
35 Command R (03-2024) 37.7% Mar 2024 Official leaderboard
36 sambalingo-thai-chat 33.6% Official leaderboard
37 Mistral 7B v0.3 32% Sep 2023 Official leaderboard
38 openthaigpt-1.0.0-7b-chat 31.1% Official leaderboard
39 Mixtral 8x7B 30% Dec 2023 Official leaderboard
40 Llama 3.1-8B 24% Jul 2024 Official leaderboard
41 sea-lion-7b-instruct 20.2% Official leaderboard
42 Llama 3-8B 19.8% Apr 2024 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Not tested since Sep 2024
Claude 3.5 Sonnet 75.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 Near the ceiling (85%) Claude 3.5 Sonnet Claude 3 Opus Qwen1.5-72B Gemini 1.0 Pro Mistral 7B v0.3 0% 20% 40% 60% 80% 100% 2024 Near the ceiling (85%) Claude 3.5 Sonnet Claude 3 Opus

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Multilingual

All 115 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Inkling 88.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%