Models Agents EvalsIndustry

AgMoDB by @mistakeknot

GPT-4o (May '24) | AgMoDB

GPT-4o (May '24)

May 13, 2024Last synced Apr 7, 2026, 4:00 PM128K context

Blended Price

$7.50/M

Input Price

$5.00/M

Output Price

$15.00/M

Speed

115 tok/s

TTFT

0.55s

Benchmark Scores

Intelligence Index

External Benchmarks

AlpacaEval 2.0 LC

How GPT-4o (May '24) Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

14.5

Coding Index

24.2

Math Index

—

MMLU Pro

0.7 / 100

GPQA Diamond

0.5 / 100

HLE

0.0 / 100

LiveCodeBench

0.3 / 100

SciCode

0.3 / 100

MATH-500

0.8 / 100

AIME

0.1 / 30

AIME 2025

— / 30

IFBench

— / 100

LCR

— / 100

Terminal-Bench Hard

— / 100

τ²-Bench

— / 100

alpacaeval

57.5 / 100

AlpacaEval 2.0 Rawalpacaeval

51.3 / 100

AA-Omniscience AccuracyPredicted

38.5 / 100

AA-Omniscience Hallucination RatePredicted

93.8 / 100

Aider PolyglotPredicted

19.6 / 100

AIME 2025Predicted

0.2 / 30

ARC-AGI-1Predicted

3.0 / 100

ARC-AGI-1 Cost per TaskPredicted

0.0

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

36.5

AA Intelligence Index (Matrix)Predicted

69.0

AA Long Context Reasoning (Matrix)Predicted

74.8

AIME 2024Predicted

48.0

AIME 2025 (Matrix)Predicted

18.8

Arena-Hard AutoPredicted

72.2

BrowseCompPredicted

50.6

BRUMO 2025Predicted

54.3

CMIMC 2025Predicted

57.2

CritPtPredicted

0.0

GPQA Diamond (Matrix)Predicted

55.8

GSM8KPredicted

87.6

HLE (Matrix)Predicted

9.2

HMMT Feb 2025Predicted

38.5

HMMT Nov 2025Predicted

86.7

HumanEvalPredicted

83.5

IFBench (Matrix)Predicted

29.0

IFEvalPredicted

83.1

IMO 2025Predicted

7.3

LiveCodeBench (Matrix)Predicted

45.7

MATH-500 (Matrix)Predicted

96.0

MathArena Apex 2025Predicted

0.3

MMLUPredicted

83.1

MMLU-Pro (Matrix)Predicted

72.6

MMMU-ProPredicted

81.0

MRCR v2Predicted

73.1

OSWorldPredicted

32.5

SimpleQAPredicted

29.5

SMT 2025Predicted

70.0

SWE-bench ProPredicted

31.5

Tau-Bench Telecom (Matrix)Predicted

93.1

Terminal-Bench 2.0Predicted

10.9

Terminal-Bench 1.0Predicted

10.5

USAMO 2025Predicted

4.2

Video-MMUPredicted

86.5

browsecompPredicted

57.1

BullshitBenchPredicted

8.4 / 100

Aider PolyglotPredicted

0.0

Apex AgentsPredicted

1.8

Arc Agi 2Predicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.9

CAD-EvalPredicted

6.6

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.3

GdpvalPredicted

0.1

GeoBenchPredicted

0.0

GSM8K (Epoch)Predicted

47.4

GSOPredicted

0.0

HellaSwagPredicted

36.0

HlePredicted

0.0

Lech Mazur WritingPredicted

6.9

METR Time HorizonsPredicted

0.1

PIQAPredicted

0.9

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.4

The Agent CompanyPredicted

1.3

TriviaQAPredicted

1.9

VPCTPredicted

0.4

WinoGrandePredicted

0.9

FrontierMathPredicted

8.0 / 100

GAIA Level 1Predicted

54.9

GAIA Level 2Predicted

39.8

GAIA Level 3Predicted

18.8

GAIAPredicted

42.1 / 100

IFBenchPredicted

0.3 / 100

LCRPredicted

0.0 / 100

LegalBenchPredicted

88.9 / 100

LiveBench CodingPredicted

60.5 / 100

LiveBench Data AnalysisPredicted

32.5 / 100

LiveBench LanguagePredicted

39.9 / 100

LiveBench MathPredicted

47.7 / 100

LiveBench OverallPredicted

34.0 / 100

LiveBench ReasoningPredicted

24.9 / 100

LongBench v2 EasyPredicted

44.3

LongBench v2 HardPredicted

43.2

LongBench v2Predicted

38.9 / 100

MathVistaPredicted

65.5 / 100

MedQA (USMLE)Predicted

75.9

MLE-benchPredicted

8.2 / 100

MMMUPredicted

64.0 / 100

MMTU Table UnderstandingPredicted

47.6 / 100

MT-BenchPredicted

8.8 / 10

NoLiMa (NIAH)Predicted

93.1 / 100

OCRBench v2Predicted

82.1 / 100

Open LLM AveragePredicted

46.8 / 100

Open LLM: BBHPredicted

70.9 / 100

Open LLM: GPQAPredicted

36.4 / 100

Open LLM: IFEvalPredicted

77.0 / 100

Open LLM: MATH Level 5Predicted

52.8 / 100

Open LLM: MMLU-PROPredicted

57.5 / 100

Open LLM: MUSRPredicted

46.0 / 100

RE-BenchPredicted

0.0

SimpleBenchPredicted

17.0 / 100

τ²-BenchPredicted

0.4 / 100

tau-bench RetailPredicted

61.6 / 100

Terminal-Bench HardPredicted

0.0 / 100

Vectara Factual ConsistencyPredicted

94.8 / 100

Vectara Hallucination RatePredicted

5.2 / 100

WebArenaPredicted

14.7 / 100

WeirdMLPredicted

19.8 / 100

BigCodeBench Completebigcodebench

61.1 / 100

BigCodeBench Instructbigcodebench

51.1 / 100

BALROGepoch_ai

0.0

OTIS Mock AIME 2024–2025epoch_ai

0.1

ScienceQAepoch_ai

0.9

simpleqasimpleqa

39.0

SWE-bench Liteswebench

18.3 / 100

SWE-bench Verifiedswebench

23.2 / 100

WildBenchwildbench

58.8