Benchmarks · /benchmarks/vals-ai-mmlu_pro

MMLU Pro

Name: MMLU Pro
Creator: Vals AI

MMLU Pro result as reported by Vals AI.

Source · Vals AI
Version · vals-ai snapshot 2026-10-03
Scores · 126

Test details

Visible tradeoffsThis is an objective signal, so it is mainly about measurable task performance rather than public taste.

source

Vals AI

metric

Accuracy (%)

judge

Objective

direction

higher better

group id

vals_mmlu_pro_current

domain

Reasoning / math / science

What it measures vs what it misses

✓ Measures

Multiple-choice academic reasoning across broad subjects.

✗ Misses

Adjacent skills outside the benchmark task mix, latency, and cost.

Why this countsIt is one of the cleaner reads on deliberate reasoning strength rather than style or popularity.Same-test ruleThis percentile only compares models inside the exact benchmark/version group shown here. It is not a universal score.What it missesIt still misses product usability, latency, and whether the model stays correct in messy real workflows.

Leaderboard · this benchmark version

#1 · Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5-1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 100%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

92.4%91.9% - 92.9%

#2 · Claude Opus 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 99.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

91.6%91.1% - 92.1%

#3 · Claude Fable 5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 98.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

91.5%91% - 92%

#4 · Gemini 3.1 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 97.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

91%90.4% - 91.5%

#5 · gemini-3.8-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.8-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 96.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.2%89.6% - 90.8%

#6 · gemini-3.7-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.7-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 96%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.1%89.5% - 90.7%

#7 · Gemini 3 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 95.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.1%89.5% - 90.7%

#8 · Claude Opus 4.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 94.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.9%89.3% - 90.5%

#9 · Claude Opus 4.8

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-8

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 93.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.6%89% - 90.2%

#10 · Gemini 3.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 92.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

89.5%88.9% - 90.1%

#11 · grok-4.6-high

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 92%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

89.4%88.8% - 90%

#12 · Qwen3.7 Max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.7-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 91.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

89.3%88.7% - 89.9%

#13 · gemini-3.6-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.6-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 90.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

89.3%88.7% - 89.9%

#14 · grok-4.5

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 89.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

89.2%88.6% - 89.8%

#15 · Claude Opus 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-6-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 88.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.1%88.2% - 90%

#16 · GPT-5.6 Sol

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-sol

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 88%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

89.1%88.5% - 89.7%

#17 · muse-spark-1.1

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 87.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

88.7%88.1% - 89.3%

#18 · qwen3.8-max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 86.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

88.6%88% - 89.2%

#19 · Gemini 3 Flash Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-flash-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 85.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

88.6%88% - 89.2%

#20 · muse-spark-1.2 (xHigh)

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 84.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

88.3%87.7% - 88.9%

#21 · GPT-5.5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 84%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

88.1%87.5% - 88.8%

#22 · kimi-k3-max

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 83.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

88%87.3% - 88.6%

#23 · claude-opus-4-1-20250805-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 82.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.9%87.3% - 88.6%

#24 · qwen3.6-plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.6-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 81.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

87.7%87% - 88.3%

#25 · kimi-k2.6

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.6

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 80.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

87.6%86.9% - 88.2%

#26 · Claude Sonnet 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 80%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.5%86.8% - 88.3%

#27 · GPT-5.4

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-2026-03-05

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 79.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

87.5%86.7% - 88.3%

#28 · claude-sonnet-4-5-20250929-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-5-20250929-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 78.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.4%86.6% - 88.1%

#29 · Claude Sonnet 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 77.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.3%86.5% - 88.2%

#30 · muse-spark

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 76.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

87.3%86.7% - 88%

#31 · claude-opus-4-5-20251101-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 76%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.3%86.5% - 88%

#32 · deepseek-v4-pro

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-pro

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 75.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

87.2%86.6% - 87.9%

#33 · Claude Opus 4.1

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 74.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.2%86.6% - 87.9%

#34 · Qwen3.5 Plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-plus-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 73.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

87.2%86.5% - 87.8%

#35 · MiniMax-M2.1

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 72.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

87%86.4% - 87.7%

#36 · deepseek-v4-pro-0813

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-pro-0813

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 72%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

87%86.3% - 87.6%

#37 · glm-5.1

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 71.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.9%86.2% - 87.6%

#38 · glm-5.3-max

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 70.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.8%86.1% - 87.4%

#39 · glm-5.2-max

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.2

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 69.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.7%86.1% - 87.4%

#40 · GPT-5.6 Terra

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-terra

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 68.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.7%86% - 87.3%

#41 · GPT-5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 68%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.5%85.9% - 87.2%

#42 · GPT-5.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.1-2025-11-13

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 67.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.4%85.7% - 87%

#43 · inkling

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 66.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Thinkingmachines.

86.3%85.6% - 87%

#44 · Grok 4.20

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.20-0309-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 65.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

86.3%85.6% - 86.9%

#45 · Gemini 3.1 Flash-Lite Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-flash-lite-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 64.8%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

86.2%85.6% - 86.9%

#46 · gpt-5.2-2025-12-11

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.2-2025-12-11

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 64%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.2%85.6% - 86.9%

#47 · deepseek-v4-flash-0731

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-flash-0731

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 63.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

86.2%85.5% - 86.9%

#48 · Claude Opus 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-20250514

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 62.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

86.2%85.5% - 86.8%

#49 · glm-5.3-flash

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 61.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.1%85.4% - 86.7%

#50 · GPT-5.6 Luna

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-luna

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 60.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86%85.4% - 86.7%

#51 · glm-5

VALS-AI · Sep 1, 2026

Source label: zai/glm-5-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 60%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86%85.4% - 86.7%

#52 · kimi-k2.5-thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.5-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 59.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

85.9%85.2% - 86.6%

#53 · Grok 4.3

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 58.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

85.8%85.2% - 86.5%

#54 · gemini-3.5-flash-lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash-lite

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 57.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

85.8%85.2% - 86.5%

#55 · Nemotron 3 Ultra 550B A55B (Reasoning)

VALS-AI · Sep 1, 2026

Source label: nvidia/nemotron-3-ultra-550b-a55b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 56.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Nvidia.

85.8%85.1% - 86.4%

#56 · o3

VALS-AI · Sep 1, 2026

Source label: openai/o3-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 56%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

85.6%84.9% - 86.3%

#57 · Claude Opus 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 55.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

85.6%84.9% - 86.3%

#58 · Inkling Small

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling-small

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 54.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Thinkingmachines.

85.6%84.9% - 86.2%

#59 · Grok 4

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-0709

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 53.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

85.3%84.6% - 86%

#60 · Qwen3 Max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3-max-2026-01-23

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 52.8%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

85%84.3% - 85.7%

#61 · deepseek-v3p2-thinking

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3p2-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 52%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

84.9%84.2% - 85.6%

#62 · mimo-v2.5-pro

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5-pro

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 51.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Xiaomi.

84.6%83.7% - 85.5%

#63 · GPT-5.4 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-mini-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 50.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

84.6%83.9% - 85.3%

#64 · qwen3.8-27b

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-27b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 49.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

84.3%83.6% - 85%

#65 · minimax-m3

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 48.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

84.2%83.5% - 84.9%

#66 · Grok 4.1 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-1-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 48%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

84.2%83.5% - 84.9%

#67 · Qwen3.5 Flash

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-flash

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 47.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

84.1%83.4% - 84.8%

#68 · Gemini 2.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-pro-exp-03-25

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 46.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

84.1%83.4% - 84.8%

#69 · Claude Sonnet 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 45.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

83.9%83.2% - 84.6%

#70 · Claude Sonnet 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

backfilledproxy backfilledBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 45.6%
Last updated: recent
Eligibility: Fallback benchmark identity is visible for context but excluded from default ranking.
Identity: benchmark proxy (0.58)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic. Backfilled from Claude Sonnet 4 via approved benchmark identity mapping map-claude-sonnet-4-5-to-4.

83.9%83.2% - 84.6%

#71 · Gemini 2.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-preview-09-2025

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 44%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

83.7%83% - 84.4%

#72 · o1

VALS-AI · Sep 1, 2026

Source label: openai/o1-2024-12-17

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 43.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

83.5%82.8% - 84.2%

#73 · DeepSeek Reasoner

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-r1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 42.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

83.2%82.3% - 84.1%

#74 · deepseek-v3p2

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3p2

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 41.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

83.1%82.3% - 83.8%

#75 · mimo-v2.5

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 40.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Xiaomi.

82.9%82.2% - 83.7%

#76 · glm-4.7

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 40%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

82.7%82% - 83.5%

#77 · Claude Sonnet 3.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-7-sonnet-20250219-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 39.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

82.7%82% - 83.5%

#78 · gpt-5-mini-2025-08-07

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-mini-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 38.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

82.2%81.5% - 83%

#79 · glm-4.6

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 37.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

82.2%81.5% - 82.9%

#80 · Ling 3.0 Flash

VALS-AI · Sep 1, 2026

Source label: ant/ling-3.0-flash-2607

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 36.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Ant.

82%81.3% - 82.7%

#81 · Grok 3 mini

VALS-AI · Sep 1, 2026

Source label: grok/grok-3-mini-fast-high-reasoning

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 36%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

81.4%80.6% - 82.1%

#82 · Qwen3 235B A22B

VALS-AI · Sep 1, 2026

Source label: fireworks/qwen3-235b-a22b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 35.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

81.2%80.5% - 82%

#83 · glm-4.5

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 34.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

81.2%80.5% - 82%

#84 · Kimi K2 Thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 33.6%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

81.1%80.3% - 81.8%

#85 · o4 mini

VALS-AI · Sep 1, 2026

Source label: openai/o4-mini-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 32.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

80.6%79.8% - 81.3%

#86 · GPT-4.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 32%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

80.5%79.7% - 81.2%

#87 · minimax-m2.7

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.7

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 31.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

80.4%79.7% - 81.2%

#88 · minimax-m2.5

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 30.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

80.1%79.3% - 80.9%

#89 · Grok 3

VALS-AI · Sep 1, 2026

Source label: grok/grok-3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 29.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

79.9%79.2% - 80.7%

#90 · Mistral Large (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2512

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 28.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

79.8%78.9% - 80.7%

#91 · Grok 4 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 28%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

79.7%78.9% - 80.5%

#92 · deepseek-v3-0324

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3-0324

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 27.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.5%78.7% - 80.3%

#93 · llama4-maverick-instruct-basic

VALS-AI · Sep 1, 2026

Source label: fireworks/llama4-maverick-instruct-basic

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 26.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.4%78.6% - 80.2%

#94 · kimi-k2-instruct

VALS-AI · Sep 1, 2026

Source label: together/moonshotai/Kimi-K2-Instruct

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 25.6%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

79.4%78.6% - 80.2%

#95 · GPT-OSS 120B

VALS-AI · Sep 1, 2026

Source label: fireworks/gpt-oss-120b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 24.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.2%78.4% - 79.9%

#96 · Gemini 2.5 Flash-Lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-lite-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 24%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

79.1%78.3% - 79.9%

#97 · Claude Haiku 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-haiku-4-5-20251001-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 23.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

78.7%77.8% - 79.6%

#98 · o3 mini

VALS-AI · Sep 1, 2026

Source label: openai/o3-mini-2025-01-31

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 22.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

78.7%77.9% - 79.5%

#99 · Claude Sonnet 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-sonnet-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 21.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

78.4%77.6% - 79.2%

#100 · Gemini 2.0 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.0-flash-001

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 20.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

77.4%76.6% - 78.2%

#101 · GPT-4.1 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-mini-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 20%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

77.2%76.4% - 78%

#102 · GPT-5.4 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-nano-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 19.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

77.2%76.3% - 78%

#103 · Grok 2

VALS-AI · Sep 1, 2026

Source label: grok/grok-2-1212

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 18.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

75.5%74.7% - 76.3%

#104 · mistral-medium-3.5

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-3.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 17.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

75.3%74.5% - 76.2%

#105 · Gemini 1.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-pro-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 16.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

75.3%74.5% - 76.1%

#106 · mistral-medium-2505

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-2505

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 16%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

75.3%74.5% - 76.1%

#107 · GPT-4o

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-2024-08-06

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 15.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

74.1%73.3% - 75%

#108 · deepseek-v3

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 14.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

73.8%73% - 74.7%

#109 · GPT-OSS 20B

VALS-AI · Sep 1, 2026

Source label: fireworks/gpt-oss-20b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 13.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

71.6%70.8% - 72.5%

#110 · llama-3-3-nemotron-super-49b-v1-42e84561

VALS-AI · Sep 1, 2026

Source label: together/langston/nim/nvidia/llama-3.3-nemotron-super-49b-v1-42e84561

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 12.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

70.8%69.9% - 71.7%

#111 · llama-3-3-70b-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.3-70B-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 12%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.9%69% - 70.7%

#112 · mistral-large-2411

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2411

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 11.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

69.7%68.8% - 70.6%

#113 · llama-4-scout-17b-16e-instruct

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-4-Scout-17B-16E-Instruct

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 10.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.6%68.8% - 70.5%

#114 · llama-3-3-nemotron-super-49b-v1-42e84561-thinking

VALS-AI · Sep 1, 2026

Source label: together/langston/nim/nvidia/llama-3.3-nemotron-super-49b-v1-42e84561-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 9.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.6%68.7% - 70.4%

#115 · command-a-03-2025

VALS-AI · Sep 1, 2026

Source label: cohere/command-a-03-2025

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 8.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Cohere.

69.2%68.3% - 70.1%

#116 · laguna-xs.2

VALS-AI · Sep 1, 2026

Source label: poolside/laguna-xs.2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Poolside.

69.1%68.1% - 70%

#117 · laguna-m.1

VALS-AI · Sep 1, 2026

Source label: poolside/laguna-m.1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 7.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Poolside.

68.8%68% - 69.7%

#118 · Magistral Medium 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-medium-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 6.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

68.7%67.8% - 69.5%

#119 · Mistral Small (Sep '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2503

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 5.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

66%65.1% - 66.9%

#120 · Gemini 1.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-flash-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 4.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

65.6%64.7% - 66.5%

#121 · Mistral Small (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2402

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

64.4%63.5% - 65.3%

#122 · Claude Haiku 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-haiku-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 3.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

64.1%63.2% - 65%

#123 · GPT-4.1 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-nano-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 2.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

63.5%62.6% - 64.4%

#124 · GPT-4o mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-mini-2024-07-18

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 1.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

62.7%61.8% - 63.6%

#125 · Magistral Small 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-small-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 0.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

62.1%61.2% - 63%

#126 · command-r-plus

VALS-AI · Sep 1, 2026

Source label: cohere/command-r-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 0%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Cohere.

44%43.1% - 44.9%

Benchmarks · /benchmarks/vals-ai-mmlu_pro

MMLU Pro

MMLU Pro result as reported by Vals AI.

Source · Vals AI
Version · vals-ai snapshot 2026-10-03
Scores · 126

Test details

Visible tradeoffsThis is an objective signal, so it is mainly about measurable task performance rather than public taste.

source

Vals AI

metric

Accuracy (%)

judge

Objective

direction

higher better

group id

vals_mmlu_pro_current

domain

Reasoning / math / science

What it measures vs what it misses

✓ Measures

Multiple-choice academic reasoning across broad subjects.

✗ Misses

Adjacent skills outside the benchmark task mix, latency, and cost.

Leaderboard · this benchmark version

#1 · Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5-1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 100%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

92.4%91.9% - 92.9%

#2 · Claude Opus 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 99.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

91.6%91.1% - 92.1%

#3 · Claude Fable 5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 98.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

91.5%91% - 92%

#4 · Gemini 3.1 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 97.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

91%90.4% - 91.5%

#5 · gemini-3.8-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.8-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 96.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.2%89.6% - 90.8%

#6 · gemini-3.7-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.7-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 96%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.1%89.5% - 90.7%

#7 · Gemini 3 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 95.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

90.1%89.5% - 90.7%

#8 · Claude Opus 4.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 94.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.9%89.3% - 90.5%

#9 · Claude Opus 4.8

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-8

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 93.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.6%89% - 90.2%

#10 · Gemini 3.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 92.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

89.5%88.9% - 90.1%

#11 · grok-4.6-high

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 92%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

89.4%88.8% - 90%

#12 · Qwen3.7 Max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.7-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 91.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

89.3%88.7% - 89.9%

#13 · gemini-3.6-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.6-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 90.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

89.3%88.7% - 89.9%

#14 · grok-4.5

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 89.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

89.2%88.6% - 89.8%

#15 · Claude Opus 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-6-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 88.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

89.1%88.2% - 90%

#16 · GPT-5.6 Sol

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-sol

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 88%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

89.1%88.5% - 89.7%

#17 · muse-spark-1.1

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 87.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

88.7%88.1% - 89.3%

#18 · qwen3.8-max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 86.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

88.6%88% - 89.2%

#19 · Gemini 3 Flash Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-flash-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 85.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

88.6%88% - 89.2%

#20 · muse-spark-1.2 (xHigh)

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 84.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

88.3%87.7% - 88.9%

#21 · GPT-5.5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 84%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

88.1%87.5% - 88.8%

#22 · kimi-k3-max

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 83.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

88%87.3% - 88.6%

#23 · claude-opus-4-1-20250805-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 82.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.9%87.3% - 88.6%

#24 · qwen3.6-plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.6-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 81.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

87.7%87% - 88.3%

#25 · kimi-k2.6

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.6

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 80.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

87.6%86.9% - 88.2%

#26 · Claude Sonnet 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 80%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.5%86.8% - 88.3%

#27 · GPT-5.4

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-2026-03-05

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 79.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

87.5%86.7% - 88.3%

#28 · claude-sonnet-4-5-20250929-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-5-20250929-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 78.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.4%86.6% - 88.1%

#29 · Claude Sonnet 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 77.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.3%86.5% - 88.2%

#30 · muse-spark

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 76.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Meta.

87.3%86.7% - 88%

#31 · claude-opus-4-5-20251101-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 76%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.3%86.5% - 88%

#32 · deepseek-v4-pro

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-pro

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 75.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

87.2%86.6% - 87.9%

#33 · Claude Opus 4.1

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 74.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

87.2%86.6% - 87.9%

#34 · Qwen3.5 Plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-plus-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 73.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

87.2%86.5% - 87.8%

#35 · MiniMax-M2.1

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 72.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

87%86.4% - 87.7%

#36 · deepseek-v4-pro-0813

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-pro-0813

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 72%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

87%86.3% - 87.6%

#37 · glm-5.1

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 71.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.9%86.2% - 87.6%

#38 · glm-5.3-max

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 70.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.8%86.1% - 87.4%

#39 · glm-5.2-max

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.2

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 69.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.7%86.1% - 87.4%

#40 · GPT-5.6 Terra

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-terra

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 68.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.7%86% - 87.3%

#41 · GPT-5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 68%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.5%85.9% - 87.2%

#42 · GPT-5.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.1-2025-11-13

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 67.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.4%85.7% - 87%

#43 · inkling

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 66.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Thinkingmachines.

86.3%85.6% - 87%

#44 · Grok 4.20

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.20-0309-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 65.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

86.3%85.6% - 86.9%

#45 · Gemini 3.1 Flash-Lite Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-flash-lite-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 64.8%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

86.2%85.6% - 86.9%

#46 · gpt-5.2-2025-12-11

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.2-2025-12-11

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 64%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86.2%85.6% - 86.9%

#47 · deepseek-v4-flash-0731

VALS-AI · Sep 1, 2026

Source label: deepseek/deepseek-v4-flash-0731

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 63.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: DeepSeek.

86.2%85.5% - 86.9%

#48 · Claude Opus 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-20250514

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 62.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

86.2%85.5% - 86.8%

#49 · glm-5.3-flash

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 61.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86.1%85.4% - 86.7%

#50 · GPT-5.6 Luna

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-luna

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 60.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

86%85.4% - 86.7%

#51 · glm-5

VALS-AI · Sep 1, 2026

Source label: zai/glm-5-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 60%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

86%85.4% - 86.7%

#52 · kimi-k2.5-thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.5-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 59.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

85.9%85.2% - 86.6%

#53 · Grok 4.3

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 58.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

85.8%85.2% - 86.5%

#54 · gemini-3.5-flash-lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash-lite

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 57.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

85.8%85.2% - 86.5%

#55 · Nemotron 3 Ultra 550B A55B (Reasoning)

VALS-AI · Sep 1, 2026

Source label: nvidia/nemotron-3-ultra-550b-a55b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 56.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Nvidia.

85.8%85.1% - 86.4%

#56 · o3

VALS-AI · Sep 1, 2026

Source label: openai/o3-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 56%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

85.6%84.9% - 86.3%

#57 · Claude Opus 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 55.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

85.6%84.9% - 86.3%

#58 · Inkling Small

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling-small

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 54.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Thinkingmachines.

85.6%84.9% - 86.2%

#59 · Grok 4

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-0709

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 53.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

85.3%84.6% - 86%

#60 · Qwen3 Max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3-max-2026-01-23

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 52.8%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

85%84.3% - 85.7%

#61 · deepseek-v3p2-thinking

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3p2-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 52%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

84.9%84.2% - 85.6%

#62 · mimo-v2.5-pro

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5-pro

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 51.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Xiaomi.

84.6%83.7% - 85.5%

#63 · GPT-5.4 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-mini-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 50.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

84.6%83.9% - 85.3%

#64 · qwen3.8-27b

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-27b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 49.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

84.3%83.6% - 85%

#65 · minimax-m3

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 48.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

84.2%83.5% - 84.9%

#66 · Grok 4.1 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-1-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 48%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

84.2%83.5% - 84.9%

#67 · Qwen3.5 Flash

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-flash

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 47.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Alibaba.

84.1%83.4% - 84.8%

#68 · Gemini 2.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-pro-exp-03-25

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 46.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

84.1%83.4% - 84.8%

#69 · Claude Sonnet 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 45.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

83.9%83.2% - 84.6%

#70 · Claude Sonnet 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

backfilledproxy backfilledBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 45.6%
Last updated: recent
Eligibility: Fallback benchmark identity is visible for context but excluded from default ranking.
Identity: benchmark proxy (0.58)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic. Backfilled from Claude Sonnet 4 via approved benchmark identity mapping map-claude-sonnet-4-5-to-4.

83.9%83.2% - 84.6%

#71 · Gemini 2.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-preview-09-2025

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 44%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

83.7%83% - 84.4%

#72 · o1

VALS-AI · Sep 1, 2026

Source label: openai/o1-2024-12-17

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 43.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

83.5%82.8% - 84.2%

#73 · DeepSeek Reasoner

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-r1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 42.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

83.2%82.3% - 84.1%

#74 · deepseek-v3p2

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3p2

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 41.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

83.1%82.3% - 83.8%

#75 · mimo-v2.5

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 40.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Xiaomi.

82.9%82.2% - 83.7%

#76 · glm-4.7

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 40%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

82.7%82% - 83.5%

#77 · Claude Sonnet 3.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-7-sonnet-20250219-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 39.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

82.7%82% - 83.5%

#78 · gpt-5-mini-2025-08-07

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-mini-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 38.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

82.2%81.5% - 83%

#79 · glm-4.6

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 37.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

82.2%81.5% - 82.9%

#80 · Ling 3.0 Flash

VALS-AI · Sep 1, 2026

Source label: ant/ling-3.0-flash-2607

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 36.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Ant.

82%81.3% - 82.7%

#81 · Grok 3 mini

VALS-AI · Sep 1, 2026

Source label: grok/grok-3-mini-fast-high-reasoning

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 36%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

81.4%80.6% - 82.1%

#82 · Qwen3 235B A22B

VALS-AI · Sep 1, 2026

Source label: fireworks/qwen3-235b-a22b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 35.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

81.2%80.5% - 82%

#83 · glm-4.5

VALS-AI · Sep 1, 2026

Source label: zai/glm-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 34.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Zhipu AI.

81.2%80.5% - 82%

#84 · Kimi K2 Thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 33.6%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Moonshot AI.

81.1%80.3% - 81.8%

#85 · o4 mini

VALS-AI · Sep 1, 2026

Source label: openai/o4-mini-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 32.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

80.6%79.8% - 81.3%

#86 · GPT-4.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 32%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

80.5%79.7% - 81.2%

#87 · minimax-m2.7

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.7

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 31.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

80.4%79.7% - 81.2%

#88 · minimax-m2.5

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 30.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: MiniMax.

80.1%79.3% - 80.9%

#89 · Grok 3

VALS-AI · Sep 1, 2026

Source label: grok/grok-3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 29.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

79.9%79.2% - 80.7%

#90 · Mistral Large (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2512

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 28.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

79.8%78.9% - 80.7%

#91 · Grok 4 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 28%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

79.7%78.9% - 80.5%

#92 · deepseek-v3-0324

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3-0324

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 27.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.5%78.7% - 80.3%

#93 · llama4-maverick-instruct-basic

VALS-AI · Sep 1, 2026

Source label: fireworks/llama4-maverick-instruct-basic

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 26.4%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.4%78.6% - 80.2%

#94 · kimi-k2-instruct

VALS-AI · Sep 1, 2026

Source label: together/moonshotai/Kimi-K2-Instruct

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 25.6%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

79.4%78.6% - 80.2%

#95 · GPT-OSS 120B

VALS-AI · Sep 1, 2026

Source label: fireworks/gpt-oss-120b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 24.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

79.2%78.4% - 79.9%

#96 · Gemini 2.5 Flash-Lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-lite-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 24%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

79.1%78.3% - 79.9%

#97 · Claude Haiku 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-haiku-4-5-20251001-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 23.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

78.7%77.8% - 79.6%

#98 · o3 mini

VALS-AI · Sep 1, 2026

Source label: openai/o3-mini-2025-01-31

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 22.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

78.7%77.9% - 79.5%

#99 · Claude Sonnet 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-sonnet-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 21.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

78.4%77.6% - 79.2%

#100 · Gemini 2.0 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.0-flash-001

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 20.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

77.4%76.6% - 78.2%

#101 · GPT-4.1 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-mini-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 20%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

77.2%76.4% - 78%

#102 · GPT-5.4 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-nano-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 19.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

77.2%76.3% - 78%

#103 · Grok 2

VALS-AI · Sep 1, 2026

Source label: grok/grok-2-1212

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 18.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: SpaceXAI.

75.5%74.7% - 76.3%

#104 · mistral-medium-3.5

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-3.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 17.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

75.3%74.5% - 76.2%

#105 · Gemini 1.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-pro-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 16.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

75.3%74.5% - 76.1%

#106 · mistral-medium-2505

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-2505

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 16%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

75.3%74.5% - 76.1%

#107 · GPT-4o

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-2024-08-06

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 15.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

74.1%73.3% - 75%

#108 · deepseek-v3

VALS-AI · Sep 1, 2026

Source label: fireworks/deepseek-v3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 14.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

73.8%73% - 74.7%

#109 · GPT-OSS 20B

VALS-AI · Sep 1, 2026

Source label: fireworks/gpt-oss-20b

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 13.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Fireworks AI.

71.6%70.8% - 72.5%

#110 · llama-3-3-nemotron-super-49b-v1-42e84561

VALS-AI · Sep 1, 2026

Source label: together/langston/nim/nvidia/llama-3.3-nemotron-super-49b-v1-42e84561

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 12.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

70.8%69.9% - 71.7%

#111 · llama-3-3-70b-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.3-70B-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 12%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.9%69% - 70.7%

#112 · mistral-large-2411

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2411

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 11.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

69.7%68.8% - 70.6%

#113 · llama-4-scout-17b-16e-instruct

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-4-Scout-17B-16E-Instruct

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 10.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.6%68.8% - 70.5%

#114 · llama-3-3-nemotron-super-49b-v1-42e84561-thinking

VALS-AI · Sep 1, 2026

Source label: together/langston/nim/nvidia/llama-3.3-nemotron-super-49b-v1-42e84561-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 9.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Together AI.

69.6%68.7% - 70.4%

#115 · command-a-03-2025

VALS-AI · Sep 1, 2026

Source label: cohere/command-a-03-2025

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 8.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Cohere.

69.2%68.3% - 70.1%

#116 · laguna-xs.2

VALS-AI · Sep 1, 2026

Source label: poolside/laguna-xs.2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Poolside.

69.1%68.1% - 70%

#117 · laguna-m.1

VALS-AI · Sep 1, 2026

Source label: poolside/laguna-m.1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 7.2%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Poolside.

68.8%68% - 69.7%

#118 · Magistral Medium 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-medium-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 6.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

68.7%67.8% - 69.5%

#119 · Mistral Small (Sep '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2503

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 5.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

66%65.1% - 66.9%

#120 · Gemini 1.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-flash-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 4.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Google.

65.6%64.7% - 66.5%

#121 · Mistral Small (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2402

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

64.4%63.5% - 65.3%

#122 · Claude Haiku 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-haiku-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 3.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Anthropic.

64.1%63.2% - 65%

#123 · GPT-4.1 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-nano-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 2.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

63.5%62.6% - 64.4%

#124 · GPT-4o mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-mini-2024-07-18

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 1.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: OpenAI.

62.7%61.8% - 63.6%

#125 · Magistral Small 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-small-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 0.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Mistral AI.

62.1%61.2% - 63%

#126 · command-r-plus

VALS-AI · Sep 1, 2026

Source label: cohere/command-r-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmlu_pro
Percentile: 0%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmlu_pro; provider: Cohere.

44%43.1% - 44.9%

MMLU Pro

Test details

What it measures vs what it misses

✓ Measures

✗ Misses

Leaderboard · this benchmark version

Loading benchmark evidence.

MMLU Pro

Test details

What it measures vs what it misses

✓ Measures

✗ Misses

Leaderboard · this benchmark version