Models Agents EvalsIndustry

AgMoDB by @mistakeknot

GPT-4o (Aug '24) | AgMoDB

GPT-4o (Aug '24)

August 6, 2024Last synced Apr 7, 2026, 4:00 PM128K context

Blended Price

$4.38/M

Input Price

$2.50/M

Output Price

$10.00/M

Speed

106 tok/s

TTFT

0.50s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How GPT-4o (Aug '24) Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

18.6

Coding Index

16.6

Math Index

—

MMLU Pro

— / 100

GPQA Diamond

0.5 / 100

HLE

0.0 / 100

LiveCodeBench

0.3 / 100

SciCode

0.3 / 100

MATH-500

0.8 / 100

AIME

0.1 / 30

AIME 2025

— / 30

IFBench

0.4 / 100

LCR

0.3 / 100

Terminal-Bench Hard

0.1 / 100

τ²-Bench

0.3 / 100

aider

23.1 / 100

AlpacaEval 2.0 LCalpacaeval

50.7 / 100

AlpacaEval 2.0 Rawalpacaeval

44.7 / 100

ARC-AGI-1arc_agi

4.5 / 100

ARC-AGI-1 Cost per Taskarc_agi

0.1

ARC-AGI-2arc_agi

0.0 / 100

ARC-AGI-2 Cost per Taskarc_agi

0.1

AA-Omniscience AccuracyPredicted

38.4 / 100

AA-Omniscience Hallucination RatePredicted

93.1 / 100

AIME 2025Predicted

0.3 / 30

BFCL (Berkeley Function Calling)Predicted

42.7

AA Intelligence Index (Matrix)Predicted

70.2

AA Long Context Reasoning (Matrix)Predicted

68.9

AIME 2024Predicted

72.6

AIME 2025 (Matrix)Predicted

43.3

Arena-Hard AutoPredicted

74.8

BrowseCompPredicted

49.6

BRUMO 2025Predicted

80.3

CMIMC 2025Predicted

60.0

CritPtPredicted

0.1

GPQA Diamond (Matrix)Predicted

72.7

GSM8KPredicted

92.6

HLE (Matrix)Predicted

10.0

HMMT Feb 2025Predicted

39.9

HMMT Nov 2025Predicted

92.3

HumanEvalPredicted

84.3

IFBench (Matrix)Predicted

34.8

IFEvalPredicted

81.8

IMO 2025Predicted

3.4

LiveCodeBench (Matrix)Predicted

54.2

MATH-500 (Matrix)Predicted

95.0

MathArena Apex 2025Predicted

2.4

MMLUPredicted

84.3

MMLU-Pro (Matrix)Predicted

78.1

MMMU-ProPredicted

81.7

MRCR v2Predicted

80.0

OSWorldPredicted

28.1

SimpleQAPredicted

18.0

SMT 2025Predicted

71.4

SWE-bench ProPredicted

22.7

Tau-Bench Telecom (Matrix)Predicted

93.3

Terminal-Bench 2.0Predicted

15.7

Terminal-Bench 1.0Predicted

12.6

USAMO 2025Predicted

5.7

Video-MMUPredicted

86.7

browsecompPredicted

44.6

Apex AgentsPredicted

1.8

BALROGPredicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.9

Chess PuzzlesPredicted

0.1

DeepResearchBenchPredicted

0.3

GSM8K (Epoch)Predicted

45.8

HellaSwagPredicted

18.0

PIQAPredicted

0.9

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.2

TriviaQAPredicted

2.4

WinoGrandePredicted

0.9

FrontierMathPredicted

8.3 / 100

LegalBenchPredicted

87.4 / 100

LiveBench CodingPredicted

61.1 / 100

LiveBench Data AnalysisPredicted

33.9 / 100

LiveBench LanguagePredicted

41.5 / 100

LiveBench MathPredicted

49.3 / 100

LiveBench OverallPredicted

35.7 / 100

LiveBench ReasoningPredicted

27.1 / 100

MedQA (USMLE)Predicted

86.1

MMLU ProPredicted

0.7 / 100

MMTU Table UnderstandingPredicted

51.9 / 100

MT-BenchPredicted

8.0 / 10

OCRBench v2Predicted

84.9 / 100

Open LLM AveragePredicted

17.3 / 100

Open LLM: BBHPredicted

27.8 / 100

Open LLM: GPQAPredicted

22.7 / 100

Open LLM: IFEvalPredicted

71.4 / 100

Open LLM: MATH Level 5Predicted

57.2 / 100

Open LLM: MMLU-PROPredicted

22.2 / 100

Open LLM: MUSRPredicted

26.7 / 100

WebArenaPredicted

16.1 / 100

WeirdMLPredicted

23.6 / 100

BigCodeBench Completebigcodebench

58.9 / 100

BigCodeBench Instructbigcodebench

48.0 / 100

BullshitBenchbullshitbench

12.0 / 100

Aider Polyglotepoch_ai

0.0

Arc Agi 2epoch_ai

0.0

CAD-Evalepoch_ai

6.5

CyBenchepoch_ai

0.1

Epoch Capabilities Indexepoch_ai

128.9

FictionLiveBenchepoch_ai

0.6

Gdpvalepoch_ai

0.1

GeoBenchepoch_ai

0.0

GSOepoch_ai

0.0

Hleepoch_ai

0.0

Lech Mazur Writingepoch_ai

8.2

METR Time Horizonsepoch_ai

1.3

OTIS Mock AIME 2024–2025epoch_ai

0.1

The Agent Companyepoch_ai

1.3

VPCTepoch_ai

0.4

EQ-Bench 3eqbench

1566.0

GAIA Level 1gaia

39.8

GAIA Level 2gaia

27.0

GAIA Level 3gaia

14.3

GAIAgaia

28.9 / 100

LongBench v2 Easylongbench2

51.2

LongBench v2 Hardlongbench2

57.4

LongBench v2longbench2

50.1 / 100

MathVistamathvista

63.8 / 100

MLE-benchmlebench

8.6 / 100

MMMUmmmu

69.1 / 100

NoLiMa (NIAH)nolima

99.3 / 100

RE-Benchrebench

0.0

SimpleBenchsimplebench

17.8 / 100

simpleqasimpleqa

40.1

SWE-bench Liteswebench

21.7 / 100

SWE-bench Verifiedswebench

24.0 / 100

tau-bench Airlinetaubench

42.0 / 100

tau-bench Retailtaubench

60.4 / 100

Vectara Factual Consistencyvectara_hallucination

90.4 / 100

Vectara Hallucination Ratevectara_hallucination

9.6 / 100

Video-MME Longvideomme

72.1

Video-MME Mediumvideomme

76.6

Video-MMEvideomme

77.2 / 100

Video-MME Shortvideomme

82.8

WildBenchwildbench

57.3