Benchmarks · /benchmarks/vals-ai-mmmu

MMMU Pro

Name: MMMU Pro
Creator: Vals AI

MMMU Pro result as reported by Vals AI.

Source · Vals AI
Version · vals-ai snapshot 2026-10-03
Scores · 84

Test details

Visible tradeoffsThis is an objective signal, so it is mainly about measurable task performance rather than public taste.

source

Vals AI

metric

Accuracy (%)

judge

Objective

direction

higher better

group id

vals_mmmu_current

domain

Vision understanding

What it measures vs what it misses

✓ Measures

Multimodal reasoning over images and prompts.

✗ Misses

Adjacent skills outside the benchmark task mix, latency, and cost.

Why this countsIt is useful when the model must read charts, UI, screenshots, or visual scenes rather than text alone.Same-test ruleThis percentile only compares models inside the exact benchmark/version group shown here. It is not a universal score.What it missesIt does not tell you whether the model can generate or edit images well.

Leaderboard · this benchmark version

#1 · Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5-1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 100%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

90.6%89.3% - 92%

#2 · Claude Opus 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 98.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

89.9%88.5% - 91.3%

#3 · Claude Fable 5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 97.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

89.3%87.8% - 90.8%

#4 · gemini-3.8-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.8-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 96.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

89.1%87.6% - 90.5%

#5 · gemini-3.7-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.7-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 95.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

89%87.5% - 90.4%

#6 · GPT-5.6 Sol

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-sol

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 94%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

88.8%87.4% - 90.3%

#7 · gemini-3.6-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.6-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 92.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.4%86.9% - 89.9%

#8 · Gemini 3.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 91.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.3%86.7% - 89.8%

#9 · GPT-5.5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 91.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

88.3%86.7% - 89.8%

#10 · Gemini 3.1 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 89.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.2%86.7% - 89.7%

#11 · kimi-k3-max

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 88%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

88.2%86.6% - 89.7%

#12 · qwen3.8-max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 86.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

88%86.5% - 89.6%

#13 · Gemini 3 Flash Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-flash-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 85.5%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

87.6%86.1% - 89.2%

#14 · Gemini 3 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 84.3%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

87.5%86% - 89.1%

#15 · GPT-5.4

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-2026-03-05

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 84.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

87.5%86% - 89.1%

#16 · muse-spark

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 81.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

87.4%85.8% - 89%

#17 · gpt-5.2-2025-12-11

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.2-2025-12-11

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 80.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

86.7%85.1% - 88.3%

#18 · muse-spark-1.1

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 79.5%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

86.6%85% - 88.2%

#19 · Claude Opus 4.8

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-8

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 79.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

86.6%85% - 88.2%

#20 · GPT-5.6 Terra

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-terra

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 77.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

86.5%84.9% - 88.1%

#21 · kimi-k2.6

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.6

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 75.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

86.3%84.7% - 87.9%

#22 · muse-spark-1.2 (xHigh)

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 74.7%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

86.1%84.5% - 87.8%

#23 · glm-5.3-flash

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 73.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Zhipu AI.

86%84.4% - 87.6%

#24 · Claude Opus 4.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 72.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

85.5%83.9% - 87.2%

#25 · GPT-5.6 Luna

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-luna

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 71.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

85%83.3% - 86.7%

#26 · kimi-k2.5-thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.5-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 69.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

84.3%82.6% - 86%

#27 · qwen3.6-plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.6-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 68.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

84.2%82.4% - 85.9%

#28 · qwen3.8-27b

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-27b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 67.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

83.9%82.2% - 85.7%

#29 · Claude Opus 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-6-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 66.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83.9%82.1% - 85.6%

#30 · gemini-3.5-flash-lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash-lite

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 65.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

83.6%81.8% - 85.3%

#31 · Claude Sonnet 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 65.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83.6%81.8% - 85.3%

#32 · Grok 4.20

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.20-0309-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 62.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

83.5%81.7% - 85.2%

#33 · GPT-5.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.1-2025-11-13

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 61.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

83.2%81.4% - 84.9%

#34 · Grok 4.3

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 60.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

83.1%81.3% - 84.8%

#35 · Claude Sonnet 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 59%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83%81.2% - 84.8%

#36 · claude-opus-4-5-20251101-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 57.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

82.9%81.2% - 84.7%

#37 · Gemini 3.1 Flash-Lite Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-flash-lite-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 56.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

82.5%80.7% - 84.3%

#38 · Qwen3.5 Flash

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-flash

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 55.4%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

81.9%80.1% - 83.7%

#39 · GPT-5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 54.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

81.5%79.7% - 83.3%

#40 · Gemini 2.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-pro-exp-03-25

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 53%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

81.3%79.5% - 83.2%

#41 · minimax-m3

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 51.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: MiniMax.

81.2%79.3% - 83%

#42 · Claude Opus 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 50.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

81.1%79.3% - 82.9%

#43 · Gemini 2.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 49.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

80.8%78.9% - 82.6%

#44 · o3

VALS-AI · Sep 1, 2026

Source label: openai/o3-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 48.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

80.4%78.5% - 82.3%

#45 · mimo-v2.5

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 47%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Xiaomi.

80%78.1% - 81.9%

#46 · o4 mini

VALS-AI · Sep 1, 2026

Source label: openai/o4-mini-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 45.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

79.7%77.8% - 81.6%

#47 · claude-sonnet-4-5-20250929-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-5-20250929-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 44.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

79.3%77.4% - 81.2%

#48 · GPT-5.4 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-mini-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 43.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

79.2%77.3% - 81.2%

#49 · gpt-5-mini-2025-08-07

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-mini-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 42.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

78.9%77% - 80.8%

#50 · inkling

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 41%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Thinkingmachines.

78.5%76.6% - 80.4%

#51 · claude-opus-4-1-20250805-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 39.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

77.5%75.5% - 79.5%

#52 · o1

VALS-AI · Sep 1, 2026

Source label: openai/o1-2024-12-17

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 38.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

77.4%75.4% - 79.4%

#53 · Grok 4

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-0709

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 37.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

76.3%74.3% - 78.3%

#54 · Gemini 2.5 Flash-Lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-lite-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 36.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

75.4%73.4% - 77.5%

#55 · Claude Sonnet 3.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-7-sonnet-20250219-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 34.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

75.1%73.1% - 77.1%

#56 · Claude Sonnet 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 33.7%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

74.9%72.9% - 77%

#57 · Claude Sonnet 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

backfilledproxy backfilledBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 33.7%
Last updated: recent
Eligibility: Fallback benchmark identity is visible for context but excluded from default ranking.
Identity: benchmark proxy (0.58)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic. Backfilled from Claude Sonnet 4 via approved benchmark identity mapping map-claude-sonnet-4-5-to-4.

74.9%72.9% - 77%

#58 · Claude Opus 4.1

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 31.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

73.7%71.6% - 75.8%

#59 · GPT-5.4 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-nano-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 30.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

73.6%71.5% - 75.7%

#60 · Claude Opus 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-20250514

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 28.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

73.3%71.2% - 75.4%

#61 · Grok 4 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 27.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

72.8%70.7% - 74.9%

#62 · Grok 4.1 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-1-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 26.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

72.7%70.6% - 74.8%

#63 · GPT-4.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 25.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

72.4%70.3% - 74.5%

#64 · llama4-maverick-instruct-basic

VALS-AI · Sep 1, 2026

Source label: fireworks/llama4-maverick-instruct-basic

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 24.1%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Fireworks AI.

71.7%69.6% - 73.8%

#65 · GPT-4.1 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-mini-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 22.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

70.5%68.4% - 72.7%

#66 · Gemini 2.0 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.0-flash-001

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 21.7%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

69.8%67.6% - 71.9%

#67 · Claude Sonnet 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-sonnet-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 20.5%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

68.8%66.6% - 71%

#68 · Mistral Large (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2512

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 19.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

66.2%64% - 68.4%

#69 · Gemini 1.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-pro-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 18.1%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

65.5%63.3% - 67.7%

#70 · Magistral Small 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-small-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 16.9%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

65.2%63% - 67.4%

#71 · Magistral Medium 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-medium-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 15.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

64.6%62.3% - 66.8%

#72 · GPT-4o

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-2024-08-06

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 14.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

64%61.7% - 66.3%

#73 · mistral-medium-2505

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-2505

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 13.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

63%60.7% - 65.2%

#74 · grok-4.5

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 12%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

61.8%59.5% - 64.1%

#75 · Mistral Small (Sep '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2503

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 10.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

60.1%57.8% - 62.4%

#76 · llama-4-scout-17b-16e-instruct

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-4-Scout-17B-16E-Instruct

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 9.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

58.8%56.4% - 61.1%

#77 · Grok 2 Vision

VALS-AI · Sep 1, 2026

Source label: grok/grok-2-vision-1212

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 8.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

57.3%54.9% - 59.6%

#78 · Gemini 1.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-flash-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 7.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

57.2%54.9% - 59.5%

#79 · GPT-4o mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-mini-2024-07-18

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

56.6%54.2% - 58.9%

#80 · GPT-4.1 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-nano-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 4.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

55.1%52.7% - 57.4%

#81 · llama-3-2-90b-vision-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 3.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

48.1%45.7% - 50.4%

#82 · Claude Haiku 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-haiku-4-5-20251001-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 2.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

46.1%43.7% - 48.4%

#83 · llama-3-2-11b-vision-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 1.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

38.8%36.5% - 41.1%

#84 · Qwen3.5 Plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-plus-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 0%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

22.8%20.8% - 24.8%

Benchmarks · /benchmarks/vals-ai-mmmu

MMMU Pro

MMMU Pro result as reported by Vals AI.

Source · Vals AI
Version · vals-ai snapshot 2026-10-03
Scores · 84

Test details

Visible tradeoffsThis is an objective signal, so it is mainly about measurable task performance rather than public taste.

source

Vals AI

metric

Accuracy (%)

judge

Objective

direction

higher better

group id

vals_mmmu_current

domain

Vision understanding

What it measures vs what it misses

✓ Measures

Multimodal reasoning over images and prompts.

✗ Misses

Adjacent skills outside the benchmark task mix, latency, and cost.

Leaderboard · this benchmark version

#1 · Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5-1

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 100%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

90.6%89.3% - 92%

#2 · Claude Opus 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 98.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

89.9%88.5% - 91.3%

#3 · Claude Fable 5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-fable-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 97.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

89.3%87.8% - 90.8%

#4 · gemini-3.8-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.8-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 96.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

89.1%87.6% - 90.5%

#5 · gemini-3.7-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.7-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 95.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

89%87.5% - 90.4%

#6 · GPT-5.6 Sol

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-sol

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 94%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

88.8%87.4% - 90.3%

#7 · gemini-3.6-flash-high

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.6-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 92.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.4%86.9% - 89.9%

#8 · Gemini 3.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 91.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.3%86.7% - 89.8%

#9 · GPT-5.5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 91.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

88.3%86.7% - 89.8%

#10 · Gemini 3.1 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 89.2%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

88.2%86.7% - 89.7%

#11 · kimi-k3-max

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 88%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

88.2%86.6% - 89.7%

#12 · qwen3.8-max

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-max

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 86.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

88%86.5% - 89.6%

#13 · Gemini 3 Flash Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-flash-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 85.5%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

87.6%86.1% - 89.2%

#14 · Gemini 3 Pro Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3-pro-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 84.3%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

87.5%86% - 89.1%

#15 · GPT-5.4

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-2026-03-05

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 84.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

87.5%86% - 89.1%

#16 · muse-spark

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 81.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

87.4%85.8% - 89%

#17 · gpt-5.2-2025-12-11

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.2-2025-12-11

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 80.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

86.7%85.1% - 88.3%

#18 · muse-spark-1.1

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_1

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 79.5%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

86.6%85% - 88.2%

#19 · Claude Opus 4.8

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-8

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 79.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

86.6%85% - 88.2%

#20 · GPT-5.6 Terra

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-terra

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 77.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

86.5%84.9% - 88.1%

#21 · kimi-k2.6

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.6

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 75.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

86.3%84.7% - 87.9%

#22 · muse-spark-1.2 (xHigh)

VALS-AI · Sep 1, 2026

Source label: meta/muse_spark_1_2

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 74.7%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Meta.

86.1%84.5% - 87.8%

#23 · glm-5.3-flash

VALS-AI · Sep 1, 2026

Source label: zai/glm-5.3-flash

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 73.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Zhipu AI.

86%84.4% - 87.6%

#24 · Claude Opus 4.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-7

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 72.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

85.5%83.9% - 87.2%

#25 · GPT-5.6 Luna

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.6-luna

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 71.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

85%83.3% - 86.7%

#26 · kimi-k2.5-thinking

VALS-AI · Sep 1, 2026

Source label: kimi/kimi-k2.5-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 69.9%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Moonshot AI.

84.3%82.6% - 86%

#27 · qwen3.6-plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.6-plus

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 68.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

84.2%82.4% - 85.9%

#28 · qwen3.8-27b

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.8-27b

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 67.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

83.9%82.2% - 85.7%

#29 · Claude Opus 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-6-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 66.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83.9%82.1% - 85.6%

#30 · gemini-3.5-flash-lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.5-flash-lite

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 65.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

83.6%81.8% - 85.3%

#31 · Claude Sonnet 4.6

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-6

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 65.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83.6%81.8% - 85.3%

#32 · Grok 4.20

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.20-0309-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 62.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

83.5%81.7% - 85.2%

#33 · GPT-5.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.1-2025-11-13

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 61.4%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

83.2%81.4% - 84.9%

#34 · Grok 4.3

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.3

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 60.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

83.1%81.3% - 84.8%

#35 · Claude Sonnet 5 (Adaptive Reasoning, Max Effort)

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 59%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

83%81.2% - 84.8%

#36 · claude-opus-4-5-20251101-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 57.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

82.9%81.2% - 84.7%

#37 · Gemini 3.1 Flash-Lite Preview

VALS-AI · Sep 1, 2026

Source label: google/gemini-3.1-flash-lite-preview

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 56.6%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

82.5%80.7% - 84.3%

#38 · Qwen3.5 Flash

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-flash

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 55.4%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

81.9%80.1% - 83.7%

#39 · GPT-5

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 54.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

81.5%79.7% - 83.3%

#40 · Gemini 2.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-pro-exp-03-25

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 53%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

81.3%79.5% - 83.2%

#41 · minimax-m3

VALS-AI · Sep 1, 2026

Source label: minimax/MiniMax-M3

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 51.8%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: MiniMax.

81.2%79.3% - 83%

#42 · Claude Opus 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-5-20251101

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 50.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

81.1%79.3% - 82.9%

#43 · Gemini 2.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 49.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

80.8%78.9% - 82.6%

#44 · o3

VALS-AI · Sep 1, 2026

Source label: openai/o3-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 48.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

80.4%78.5% - 82.3%

#45 · mimo-v2.5

VALS-AI · Sep 1, 2026

Source label: xiaomi/mimo-v2.5

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 47%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Xiaomi.

80%78.1% - 81.9%

#46 · o4 mini

VALS-AI · Sep 1, 2026

Source label: openai/o4-mini-2025-04-16

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 45.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

79.7%77.8% - 81.6%

#47 · claude-sonnet-4-5-20250929-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-5-20250929-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 44.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

79.3%77.4% - 81.2%

#48 · GPT-5.4 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-mini-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 43.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

79.2%77.3% - 81.2%

#49 · gpt-5-mini-2025-08-07

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5-mini-2025-08-07

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 42.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

78.9%77% - 80.8%

#50 · inkling

VALS-AI · Sep 1, 2026

Source label: thinkingmachines/inkling

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 41%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Thinkingmachines.

78.5%76.6% - 80.4%

#51 · claude-opus-4-1-20250805-thinking

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 39.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

77.5%75.5% - 79.5%

#52 · o1

VALS-AI · Sep 1, 2026

Source label: openai/o1-2024-12-17

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 38.6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

77.4%75.4% - 79.4%

#53 · Grok 4

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-0709

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 37.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

76.3%74.3% - 78.3%

#54 · Gemini 2.5 Flash-Lite

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.5-flash-lite-preview-09-2025-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 36.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

75.4%73.4% - 77.5%

#55 · Claude Sonnet 3.7

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-7-sonnet-20250219-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 34.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

75.1%73.1% - 77.1%

#56 · Claude Sonnet 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 33.7%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

74.9%72.9% - 77%

#57 · Claude Sonnet 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-sonnet-4-20250514-thinking

backfilledproxy backfilledBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 33.7%
Last updated: recent
Eligibility: Fallback benchmark identity is visible for context but excluded from default ranking.
Identity: benchmark proxy (0.58)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic. Backfilled from Claude Sonnet 4 via approved benchmark identity mapping map-claude-sonnet-4-5-to-4.

74.9%72.9% - 77%

#58 · Claude Opus 4.1

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-1-20250805

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 31.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

73.7%71.6% - 75.8%

#59 · GPT-5.4 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-5.4-nano-2026-03-17

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 30.1%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

73.6%71.5% - 75.7%

#60 · Claude Opus 4

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-opus-4-20250514

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 28.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

73.3%71.2% - 75.4%

#61 · Grok 4 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 27.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

72.8%70.7% - 74.9%

#62 · Grok 4.1 Fast

VALS-AI · Sep 1, 2026

Source label: grok/grok-4-1-fast-reasoning

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 26.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

72.7%70.6% - 74.8%

#63 · GPT-4.1

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 25.3%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

72.4%70.3% - 74.5%

#64 · llama4-maverick-instruct-basic

VALS-AI · Sep 1, 2026

Source label: fireworks/llama4-maverick-instruct-basic

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 24.1%
Last updated: recent
Eligibility: benchmark_derived_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Fireworks AI.

71.7%69.6% - 73.8%

#65 · GPT-4.1 mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-mini-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 22.9%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

70.5%68.4% - 72.7%

#66 · Gemini 2.0 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-2.0-flash-001

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 21.7%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

69.8%67.6% - 71.9%

#67 · Claude Sonnet 3.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-3-5-sonnet-20241022

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 20.5%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

68.8%66.6% - 71%

#68 · Mistral Large (Feb '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-large-2512

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 19.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

66.2%64% - 68.4%

#69 · Gemini 1.5 Pro

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-pro-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 18.1%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

65.5%63.3% - 67.7%

#70 · Magistral Small 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-small-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 16.9%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

65.2%63% - 67.4%

#71 · Magistral Medium 1.2

VALS-AI · Sep 1, 2026

Source label: mistralai/magistral-medium-2509

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 15.7%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

64.6%62.3% - 66.8%

#72 · GPT-4o

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-2024-08-06

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 14.5%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

64%61.7% - 66.3%

#73 · mistral-medium-2505

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-medium-2505

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 13.3%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

63%60.7% - 65.2%

#74 · grok-4.5

VALS-AI · Sep 1, 2026

Source label: grok/grok-4.5

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 12%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

61.8%59.5% - 64.1%

#75 · Mistral Small (Sep '24)

VALS-AI · Sep 1, 2026

Source label: mistralai/mistral-small-2503

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 10.8%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Mistral AI.

60.1%57.8% - 62.4%

#76 · llama-4-scout-17b-16e-instruct

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-4-Scout-17B-16E-Instruct

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 9.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

58.8%56.4% - 61.1%

#77 · Grok 2 Vision

VALS-AI · Sep 1, 2026

Source label: grok/grok-2-vision-1212

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 8.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: SpaceXAI.

57.3%54.9% - 59.6%

#78 · Gemini 1.5 Flash

VALS-AI · Sep 1, 2026

Source label: google/gemini-1.5-flash-002

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 7.2%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Google.

57.2%54.9% - 59.5%

#79 · GPT-4o mini

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4o-mini-2024-07-18

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 6%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

56.6%54.2% - 58.9%

#80 · GPT-4.1 nano

VALS-AI · Sep 1, 2026

Source label: openai/gpt-4.1-nano-2025-04-14

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 4.8%
Last updated: recent
Eligibility: historical_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: OpenAI.

55.1%52.7% - 57.4%

#81 · llama-3-2-90b-vision-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 3.6%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

48.1%45.7% - 50.4%

#82 · Claude Haiku 4.5

VALS-AI · Sep 1, 2026

Source label: anthropic/claude-haiku-4-5-20251001-thinking

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 2.4%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Anthropic.

46.1%43.7% - 48.4%

#83 · llama-3-2-11b-vision-instruct-turbo

VALS-AI · Sep 1, 2026

Source label: together/meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo

verified runtimeexact alias

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 1.2%
Last updated: recent
Eligibility: headline eligible
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Together AI.

38.8%36.5% - 41.1%

#84 · Qwen3.5 Plus

VALS-AI · Sep 1, 2026

Source label: alibaba/qwen3.5-plus-thinking

verified runtimeexact aliasBackground only

Raw row drilldownsource, percentile, eligibility

Source URL: https://www.vals.ai/benchmarks/mmmu
Percentile: 0%
Last updated: recent
Eligibility: preview_model
Identity: provider alias (0.92)

Parsed from Vals AI BenchmarkView overall scores. Vals slug: mmmu; provider: Alibaba.

22.8%20.8% - 24.8%

MMMU Pro

Test details

What it measures vs what it misses

✓ Measures

✗ Misses

Leaderboard · this benchmark version

Loading benchmark evidence.

MMMU Pro

Test details

What it measures vs what it misses

✓ Measures

✗ Misses

Leaderboard · this benchmark version