Models Agents EvalsIndustry

AgMoDB by @mistakeknot

Claude 3.5 Sonnet (Oct '24) | AgMoDB

Claude 3.5 Sonnet (Oct '24)

October 22, 2024Last synced Apr 7, 2026, 4:00 PM

Blended Price

$6.00/M

Input Price

$3.00/M

Output Price

$15.00/M

Speed

0 tok/s

TTFT

0.00s

Benchmark Scores

Intelligence Index

External Benchmarks

Aider Polyglot

How Claude 3.5 Sonnet (Oct '24) Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

15.9

Coding Index

30.2

Math Index

—

MMLU Pro

0.8 / 100

GPQA Diamond

0.6 / 100

HLE

0.0 / 100

LiveCodeBench

0.4 / 100

SciCode

0.4 / 100

MATH-500

0.8 / 100

AIME

0.2 / 30

AIME 2025

— / 30

IFBench

— / 100

LCR

— / 100

Terminal-Bench Hard

— / 100

τ²-Bench

— / 100

aider

64.0 / 100

AlpacaEval 2.0 LCalpacaeval

52.4 / 100

AlpacaEval 2.0 Rawalpacaeval

40.6 / 100

AA-Omniscience AccuracyPredicted

39.6 / 100

AA-Omniscience Hallucination RatePredicted

92.6 / 100

AIME 2025Predicted

0.4 / 30

ARC-AGI-1Predicted

5.1 / 100

ARC-AGI-1 Cost per TaskPredicted

0.0

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

43.3

AA Intelligence Index (Matrix)Predicted

68.2

AA Long Context Reasoning (Matrix)Predicted

70.3

AIME 2024Predicted

80.1

AIME 2025 (Matrix)Predicted

68.0

Arena-Hard AutoPredicted

73.9

BrowseCompPredicted

56.0

BRUMO 2025Predicted

77.6

CMIMC 2025Predicted

69.3

CritPtPredicted

0.2

GPQA Diamond (Matrix)Predicted

71.2

GSM8KPredicted

92.8

HLE (Matrix)Predicted

13.3

HMMT Feb 2025Predicted

53.0

HMMT Nov 2025Predicted

94.4

HumanEvalPredicted

91.8

IFBench (Matrix)Predicted

39.0

IFEvalPredicted

82.2

IMO 2025Predicted

11.3

LiveCodeBench (Matrix)Predicted

68.2

MATH-500 (Matrix)Predicted

95.5

MathArena Apex 2025Predicted

5.0

MMLUPredicted

87.4

MMLU-Pro (Matrix)Predicted

78.4

MMMU-ProPredicted

81.3

MRCR v2Predicted

72.7

OSWorldPredicted

36.6

SimpleQAPredicted

23.5

SMT 2025Predicted

77.9

SWE-bench ProPredicted

34.9

Tau-Bench Telecom (Matrix)Predicted

95.0

Terminal-Bench 2.0Predicted

18.8

Terminal-Bench 1.0Predicted

16.1

USAMO 2025Predicted

7.6

Video-MMUPredicted

86.5

browsecompPredicted

52.1

Apex AgentsPredicted

2.0

Arc Agi 2Predicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.8

Chess PuzzlesPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.4

GdpvalPredicted

0.2

GSM8K (Epoch)Predicted

14.6

HellaSwagPredicted

8.6

PIQAPredicted

0.9

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.2

TriviaQAPredicted

3.5

WinoGrandePredicted

0.8

FrontierMathPredicted

11.1 / 100

IFBenchPredicted

0.4 / 100

LCRPredicted

0.0 / 100

LegalBenchPredicted

86.2 / 100

LiveBench CodingPredicted

64.2 / 100

LiveBench Data AnalysisPredicted

39.7 / 100

LiveBench LanguagePredicted

48.9 / 100

LiveBench MathPredicted

57.1 / 100

LiveBench OverallPredicted

42.3 / 100

LiveBench ReasoningPredicted

35.5 / 100

MedQA (USMLE)Predicted

88.7

MMTU Table UnderstandingPredicted

54.2 / 100

MT-BenchPredicted

7.9 / 10

OCRBench v2Predicted

83.6 / 100

Open LLM AveragePredicted

14.7 / 100

Open LLM: BBHPredicted

25.9 / 100

Open LLM: GPQAPredicted

23.3 / 100

Open LLM: IFEvalPredicted

65.6 / 100

Open LLM: MATH Level 5Predicted

13.8 / 100

Open LLM: MMLU-PROPredicted

19.9 / 100

Open LLM: MUSRPredicted

28.2 / 100

τ²-BenchPredicted

0.1 / 100

Terminal-Bench HardPredicted

0.1 / 100

Vectara Factual ConsistencyPredicted

93.5 / 100

Vectara Hallucination RatePredicted

6.5 / 100

WebArenaPredicted

8.0 / 100

BigCodeBench Completebigcodebench

57.5 / 100

BigCodeBench Instructbigcodebench

44.6 / 100

BullshitBenchbullshitbench

45.0 / 100

Aider Polyglotepoch_ai

0.1

BALROGepoch_ai

0.0

CAD-Evalepoch_ai

7.7

CyBenchepoch_ai

0.2

Epoch Capabilities Indexepoch_ai

130.0

GeoBenchepoch_ai

0.0

GSOepoch_ai

0.0

Hleepoch_ai

0.0

Lech Mazur Writingepoch_ai

8.0

METR Time Horizonsepoch_ai

0.5

OTIS Mock AIME 2024–2025epoch_ai

0.1

The Agent Companyepoch_ai

6.3

Training Computeepoch_ai

2.700000000000001e+25

VPCTepoch_ai

0.3

EQ-Bench 3eqbench

1427.0

GAIA Level 1gaia

78.5

GAIA Level 2gaia

64.8

GAIA Level 3gaia

40.8

GAIAgaia

65.1 / 100

LongBench v2 Easylongbench2

46.7

LongBench v2 Hardlongbench2

46.9

LongBench v2longbench2

41.0 / 100

MathVistamathvista

67.7 / 100

MLE-benchmlebench

7.6 / 100

MMMUmmmu

68.3 / 100

NoLiMa (NIAH)nolima

87.6 / 100

RE-Benchrebench

50.2

FORTRESSseal

13.0

SimpleBenchsimplebench

27.5 / 100

simpleqasimpleqa

28.9

SWE-bench Liteswebench

23.0 / 100

SWE-bench Verifiedswebench

33.6 / 100

tau-bench Airlinetaubench

36.0 / 100

tau-bench Retailtaubench

62.6 / 100

Video-MME Longvideomme

54.7

Video-MME Mediumvideomme

60.1

Video-MMEvideomme

62.9 / 100

Video-MME Shortvideomme

73.5

WeirdMLweirdml

31.0 / 100

WildBenchwildbench

54.5