Models Agents EvalsIndustry

AgMoDB by @mistakeknot

GPT-4o mini | AgMoDB

GPT-4o mini

July 18, 2024Last synced Apr 7, 2026, 4:00 PM128K context

Blended Price

$0.26/M

Input Price

$0.15/M

Output Price

$0.60/M

Speed

52 tok/s

TTFT

0.49s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How GPT-4o mini Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

12.6

Coding Index

—

Math Index

14.7

MMLU Pro

0.6 / 100

GPQA Diamond

0.4 / 100

HLE

0.0 / 100

LiveCodeBench

0.2 / 100

SciCode

0.2 / 100

MATH-500

0.8 / 100

AIME

0.1 / 30

AIME 2025

0.1 / 30

IFBench

0.3 / 100

LCR

— / 100

Terminal-Bench Hard

— / 100

τ²-Bench

— / 100

aider

3.6 / 100

AlpacaEval 2.0 LCalpacaeval

50.7 / 100

AlpacaEval 2.0 Rawalpacaeval

44.7 / 100

ARC-AGI-2arc_agi

0.0 / 100

ARC-AGI-2 Cost per Taskarc_agi

0.0

AA-Omniscience AccuracyPredicted

39.3 / 100

AA-Omniscience Hallucination RatePredicted

40.3 / 100

ARC-AGI-1Predicted

0.6 / 100

ARC-AGI-1 Cost per TaskPredicted

57.1

BFCL (Berkeley Function Calling)Predicted

31.9

AA Intelligence Index (Matrix)Predicted

74.3

AA Long Context Reasoning (Matrix)Predicted

87.2

AIME 2024Predicted

14.7

AIME 2025 (Matrix)Predicted

3.5

Arena-Hard AutoPredicted

61.4

BrowseCompPredicted

76.8

BRUMO 2025Predicted

78.7

CMIMC 2025Predicted

53.6

CritPtPredicted

0.0

GPQA Diamond (Matrix)Predicted

52.6

GSM8KPredicted

87.4

HLE (Matrix)Predicted

46.2

HMMT Feb 2025Predicted

14.4

HMMT Nov 2025Predicted

86.5

HumanEvalPredicted

75.9

IFBench (Matrix)Predicted

12.6

IFEvalPredicted

79.3

IMO 2025Predicted

6.0

LiveCodeBench (Matrix)Predicted

28.5

MATH-500 (Matrix)Predicted

92.9

MathArena Apex 2025Predicted

33.1

MMLUPredicted

77.4

MMLU-Pro (Matrix)Predicted

62.5

MMMU-ProPredicted

79.8

MRCR v2Predicted

88.5

OSWorldPredicted

36.1

SimpleQAPredicted

12.0

SMT 2025Predicted

67.1

SWE-bench ProPredicted

32.0

Tau-Bench Telecom (Matrix)Predicted

93.0

Terminal-Bench 2.0Predicted

10.9

Terminal-Bench 1.0Predicted

8.0

USAMO 2025Predicted

1.1

Video-MMUPredicted

86.0

browsecompPredicted

49.5

Apex AgentsPredicted

1.5

BIG-Bench HardPredicted

3.0

CAD-EvalPredicted

9.4

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.0

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.2

GdpvalPredicted

0.1

GSOPredicted

0.0

HellaSwagPredicted

40.2

HlePredicted

0.0

METR Time HorizonsPredicted

0.1

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.2

The Agent CompanyPredicted

0.9

TriviaQAPredicted

1.3

WinoGrandePredicted

0.8

FrontierMathPredicted

7.8 / 100

LCRPredicted

0.0 / 100

LegalBenchPredicted

83.1 / 100

LiveBench CodingPredicted

59.8 / 100

LiveBench Data AnalysisPredicted

27.7 / 100

LiveBench LanguagePredicted

54.5 / 100

LiveBench MathPredicted

48.7 / 100

LiveBench OverallPredicted

44.1 / 100

LiveBench ReasoningPredicted

40.6 / 100

MathVistaPredicted

52.9 / 100

MLE-benchPredicted

8.4 / 100

MT-BenchPredicted

8.6 / 10

Open LLM AveragePredicted

59.8 / 100

Open LLM: BBHPredicted

75.0 / 100

Open LLM: GPQAPredicted

36.9 / 100

Open LLM: IFEvalPredicted

86.9 / 100

Open LLM: MATH Level 5Predicted

67.3 / 100

Open LLM: MMLU-PROPredicted

63.9 / 100

Open LLM: MUSRPredicted

46.0 / 100

RE-BenchPredicted

0.0

SWE-bench LitePredicted

6.5 / 100

SWE-bench VerifiedPredicted

19.2 / 100

τ²-BenchPredicted

0.3 / 100

tau-bench RetailPredicted

65.2 / 100

Terminal-Bench HardPredicted

0.0 / 100

Vectara Factual ConsistencyPredicted

95.2 / 100

Vectara Hallucination RatePredicted

4.8 / 100

WebArenaPredicted

11.8 / 100

BigCodeBench Completebigcodebench

57.4 / 100

BigCodeBench Instructbigcodebench

46.1 / 100

BullshitBenchbullshitbench

2.0 / 100

Aider Polyglotepoch_ai

0.0

Arc Agi 2epoch_ai

0.0

BALROGepoch_ai

0.0

BoolQepoch_ai

0.9

Epoch Capabilities Indexepoch_ai

127.1

GeoBenchepoch_ai

0.0

GSM8K (Epoch)epoch_ai

8.0

Lech Mazur Writingepoch_ai

6.7

OTIS Mock AIME 2024–2025epoch_ai

0.1

PIQAepoch_ai

0.9

VPCTepoch_ai

0.3

GAIA Level 1gaia

7.5

GAIA Level 2gaia

4.4

GAIA Level 3gaia

0.0

GAIAgaia

4.7 / 100

LongBench v2 Easylongbench2

32.4

LongBench v2 Hardlongbench2

31.1

LongBench v2longbench2

29.3 / 100

MedQA (USMLE)medqa

72.4

MMMUmmmu

59.4 / 100

MMTU Table Understandingmmtu

40.0 / 100

NoLiMa (NIAH)nolima

84.9 / 100

OCRBench v2open-vlm-leaderboard

78.5 / 100

OpenRouter Token Volumeopenrouter_rankings

664000000000.0

OpenRouter Weekly Trendopenrouter_rankings

226.0

SimpleBenchsimplebench

10.7 / 100

simpleqasimpleqa

9.5

tau-bench Airlinetaubench

22.5 / 100

Video-MME Longvideomme

63.4

Video-MME Mediumvideomme

68.3

Video-MMEvideomme

68.9 / 100

Video-MME Shortvideomme

74.9

WeirdMLweirdml

11.8 / 100

WildBenchwildbench

57.3