Models Agents EvalsIndustry

AgMoDB by @mistakeknot

o3 | AgMoDB

o3

April 16, 2025Last synced Apr 7, 2026, 4:00 PM200K context

Blended Price

$3.50/M

Input Price

$2.00/M

Output Price

$8.00/M

Speed

127 tok/s

TTFT

7.56s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How o3 Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

38.4

Coding Index

38.4

Math Index

88.3

MMLU Pro

0.9 / 100

GPQA Diamond

0.8 / 100

HLE

0.2 / 100

LiveCodeBench

0.8 / 100

SciCode

0.4 / 100

MATH-500

1.0 / 100

AIME

0.9 / 30

AIME 2025

0.9 / 30

IFBench

0.7 / 100

LCR

0.7 / 100

Terminal-Bench Hard

0.4 / 100

τ²-Bench

0.8 / 100

aider

76.9 / 100

AlpacaEval 2.0 LCalpacaeval

76.9 / 100

AlpacaEval 2.0 Rawalpacaeval

69.1 / 100

ARC-AGI-1arc_agi

60.8 / 100

ARC-AGI-1 Cost per Taskarc_agi

0.5

ARC-AGI-2arc_agi

6.5 / 100

ARC-AGI-2 Cost per Taskarc_agi

0.8

ARC-AGI-1benchmark_matrix

75.7 / 100

ARC-AGI-2benchmark_matrix

6.5 / 100

AA Long Context Reasoning (Matrix)benchmark_matrix

69.0

AIME 2024benchmark_matrix

96.7

AIME 2025 (Matrix)benchmark_matrix

88.9

Arena-Hard Autobenchmark_matrix

85.9

BrowseCompbenchmark_matrix

49.7

BRUMO 2025benchmark_matrix

95.8

CMIMC 2025benchmark_matrix

79.4

Codeforces Ratingbenchmark_matrix

2706.0

CritPtbenchmark_matrix

1.4

GPQA Diamond (Matrix)benchmark_matrix

87.7

HLE (Matrix)benchmark_matrix

19.8

HMMT Feb 2025benchmark_matrix

77.5

HumanEvalbenchmark_matrix

88.4

IFBench (Matrix)benchmark_matrix

69.3

IFEvalbenchmark_matrix

92.1

IMO 2025benchmark_matrix

16.7

LiveCodeBench (Matrix)benchmark_matrix

75.8

MATH-500 (Matrix)benchmark_matrix

99.2

MMLUbenchmark_matrix

93.3

MMLU-Pro (Matrix)benchmark_matrix

85.9

MMMU-Probenchmark_matrix

75.0

SimpleQAbenchmark_matrix

48.6

SMT 2025benchmark_matrix

87.7

Terminal-Bench 1.0benchmark_matrix

30.2

USAMO 2025benchmark_matrix

21.9

browsecompbenchmark_matrix

49.7

FrontierMathbenchmark_matrix

10.0 / 100

MMMUbenchmark_matrix

80.1 / 100

simpleqabenchmark_matrix

48.6

SWE-bench Verifiedbenchmark_matrix

69.1 / 100

AA-Omniscience AccuracyPredicted

40.9 / 100

AA-Omniscience Hallucination RatePredicted

97.2 / 100

BigCodeBench CompletePredicted

59.9 / 100

BigCodeBench InstructPredicted

55.9 / 100

AA Intelligence Index (Matrix)Predicted

65.3

GSM8KPredicted

96.1

HMMT Nov 2025Predicted

94.2

MathArena Apex 2025Predicted

2.7

MRCR v2Predicted

77.1

OSWorldPredicted

48.6

SWE-bench ProPredicted

43.5

Tau-Bench Telecom (Matrix)Predicted

97.6

Terminal-Bench 2.0Predicted

40.4

Video-MMUPredicted

86.5

Apex AgentsPredicted

2.7

BALROGPredicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.7

Chess PuzzlesPredicted

0.2

CyBenchPredicted

0.4

GSM8K (Epoch)Predicted

0.3

HellaSwagPredicted

0.0

PIQAPredicted

0.8

PosttrainbenchPredicted

0.0

The Agent CompanyPredicted

2.8

TriviaQAPredicted

48.1

WinoGrandePredicted

0.8

LegalBenchPredicted

85.4 / 100

LiveBench CodingPredicted

76.0 / 100

LiveBench Data AnalysisPredicted

50.3 / 100

LiveBench LanguagePredicted

68.6 / 100

LiveBench MathPredicted

62.1 / 100

LiveBench OverallPredicted

50.3 / 100

LiveBench ReasoningPredicted

63.2 / 100

LongBench v2 EasyPredicted

56.9

LongBench v2 HardPredicted

47.8

LongBench v2Predicted

43.4 / 100

MathVistaPredicted

59.9 / 100

MT-BenchPredicted

8.3 / 10

NoLiMa (NIAH)Predicted

94.4 / 100

OCRBench v2Predicted

85.0 / 100

SWE-bench LitePredicted

68.6 / 100

tau-bench RetailPredicted

86.9 / 100

Vectara Factual ConsistencyPredicted

89.6 / 100

Vectara Hallucination RatePredicted

10.4 / 100

WebArenaPredicted

1.5 / 100

WildBenchPredicted

45.6

BFCL (Berkeley Function Calling)bfcl

63.0

browsecompbrowsecomp

49.7

BullshitBenchbullshitbench

26.0 / 100

Aider Polyglotepoch_ai

13.8

Arc Agi 2epoch_ai

0.0

CAD-Evalepoch_ai

4.0

DeepResearchBenchepoch_ai

0.5

Epoch Capabilities Indexepoch_ai

147.2

FictionLiveBenchepoch_ai

0.8

Gdpvalepoch_ai

0.3

GeoBenchepoch_ai

0.0

GSOepoch_ai

0.1

Hleepoch_ai

0.2

Lech Mazur Writingepoch_ai

8.4

METR Time Horizonsepoch_ai

30.0

OSWorld (Epoch)epoch_ai

9.1

OTIS Mock AIME 2024–2025epoch_ai

0.8

SimpleQA Verified (Epoch)epoch_ai

0.5

VPCTepoch_ai

0.5

EQ-Bench 3eqbench

1665.0

GAIA Level 1gaia

36.6

GAIA Level 2gaia

18.2

GAIA Level 3gaia

6.1

GAIAgaia

21.9 / 100

MedQA (USMLE)medqa

96.1

MLE-benchmlebench

31.6 / 100

MMMUmmmu

82.9 / 100

MMTU Table Understandingmmtu

69.1 / 100

Open LLM Averageopen_llm_leaderboard

4.3 / 100

Open LLM: BBHopen_llm_leaderboard

29.4 / 100

Open LLM: GPQAopen_llm_leaderboard

25.8 / 100

Open LLM: IFEvalopen_llm_leaderboard

15.1 / 100

Open LLM: MATH Level 5open_llm_leaderboard

1.0 / 100

Open LLM: MMLU-PROopen_llm_leaderboard

11.6 / 100

Open LLM: MUSRopen_llm_leaderboard

36.2 / 100

RE-Benchrebench

72.3

EnigmaEvalseal

13.1

PropensityBenchseal

10.5

SimpleBenchsimplebench

53.1 / 100

simpleqasimpleqa

49.4

SWE-bench Verifiedswebench

58.4 / 100

WeirdMLweirdml

52.4 / 100