Models Agents EvalsIndustry

AgMoDB by @mistakeknot

Mistral Large 2 (Jul '24) | AgMoDB

Mistral Large 2 (Jul '24)

July 24, 2024Last synced Apr 7, 2026, 4:00 PM131K context

Blended Price

$3.00/M

Input Price

$2.00/M

Output Price

$6.00/M

Speed

0 tok/s

TTFT

0.00s

Benchmark Scores

Intelligence Index

External Benchmarks

AlpacaEval 2.0 LC

How Mistral Large 2 (Jul '24) Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

13.0

Coding Index

—

Math Index

0.0

MMLU Pro

0.7 / 100

GPQA Diamond

0.5 / 100

HLE

0.0 / 100

LiveCodeBench

0.3 / 100

SciCode

0.3 / 100

MATH-500

0.7 / 100

AIME

0.1 / 30

AIME 2025

0.0 / 30

IFBench

0.3 / 100

LCR

0.0 / 100

Terminal-Bench Hard

— / 100

τ²-Bench

0.3 / 100

alpacaeval

32.7 / 100

AlpacaEval 2.0 Rawalpacaeval

21.4 / 100

AA-Omniscience AccuracyPredicted

42.1 / 100

AA-Omniscience Hallucination RatePredicted

88.2 / 100

Aider PolyglotPredicted

10.9 / 100

ARC-AGI-1Predicted

3.7 / 100

ARC-AGI-1 Cost per TaskPredicted

0.2

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

29.3

AA Intelligence Index (Matrix)Predicted

72.2

AA Long Context Reasoning (Matrix)Predicted

70.7

AIME 2024Predicted

22.2

AIME 2025 (Matrix)Predicted

14.5

Arena-Hard AutoPredicted

51.2

BrowseCompPredicted

25.8

BRUMO 2025Predicted

80.8

CMIMC 2025Predicted

69.8

CritPtPredicted

0.2

GPQA Diamond (Matrix)Predicted

53.5

GSM8KPredicted

85.6

HLE (Matrix)Predicted

39.6

HMMT Feb 2025Predicted

27.5

HMMT Nov 2025Predicted

89.1

HumanEvalPredicted

78.3

IFBench (Matrix)Predicted

25.7

IFEvalPredicted

75.5

IMO 2025Predicted

11.3

LiveCodeBench (Matrix)Predicted

31.8

MATH-500 (Matrix)Predicted

91.9

MathArena Apex 2025Predicted

29.8

MMLUPredicted

79.2

MMLU-Pro (Matrix)Predicted

65.9

MMMU-ProPredicted

78.9

MRCR v2Predicted

89.9

OSWorldPredicted

34.4

SimpleQAPredicted

19.1

SMT 2025Predicted

77.3

SWE-bench ProPredicted

27.6

Tau-Bench Telecom (Matrix)Predicted

38.4

Terminal-Bench 2.0Predicted

22.1

Terminal-Bench 1.0Predicted

12.2

USAMO 2025Predicted

4.1

Video-MMUPredicted

85.5

browsecompPredicted

73.8

Aider PolyglotPredicted

0.0

Apex AgentsPredicted

1.7

Arc Agi 2Predicted

0.0

BALROGPredicted

0.0

BIG-Bench HardPredicted

3.0

BoolQPredicted

0.9

CAD-EvalPredicted

10.0

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.3

GdpvalPredicted

0.1

GeoBenchPredicted

0.0

GSM8K (Epoch)Predicted

6.8

GSOPredicted

60.0

HellaSwagPredicted

9.5

HlePredicted

0.0

METR Time HorizonsPredicted

0.2

PIQAPredicted

0.9

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.4

The Agent CompanyPredicted

0.9

TriviaQAPredicted

2.4

VPCTPredicted

0.3

WinoGrandePredicted

0.8

FrontierMathPredicted

13.0 / 100

GAIA Level 1Predicted

14.3

GAIA Level 2Predicted

2.9

GAIA Level 3Predicted

0.0

GAIAPredicted

8.7 / 100

LegalBenchPredicted

70.5 / 100

LiveBench CodingPredicted

65.2 / 100

LiveBench Data AnalysisPredicted

40.8 / 100

LiveBench LanguagePredicted

50.5 / 100

LiveBench MathPredicted

54.5 / 100

LiveBench OverallPredicted

43.1 / 100

LiveBench ReasoningPredicted

35.9 / 100

MathVistaPredicted

46.4 / 100

MedQA (USMLE)Predicted

67.1

MLE-benchPredicted

18.2 / 100

MMMUPredicted

54.9 / 100

MMTU Table UnderstandingPredicted

43.9 / 100

MT-BenchPredicted

8.2 / 10

OCRBench v2Predicted

75.8 / 100

Open LLM AveragePredicted

40.8 / 100

Open LLM: BBHPredicted

65.1 / 100

Open LLM: GPQAPredicted

34.1 / 100

Open LLM: IFEvalPredicted

77.8 / 100

Open LLM: MATH Level 5Predicted

41.5 / 100

Open LLM: MMLU-PROPredicted

50.2 / 100

Open LLM: MUSRPredicted

43.6 / 100

RE-BenchPredicted

0.5

SimpleBenchPredicted

18.4 / 100

simpleqaPredicted

17.0

SWE-bench LitePredicted

7.8 / 100

SWE-bench VerifiedPredicted

26.3 / 100

tau-bench RetailPredicted

68.5 / 100

Terminal-Bench HardPredicted

0.0 / 100

Vectara Factual ConsistencyPredicted

91.8 / 100

Vectara Hallucination RatePredicted

8.2 / 100

WebArenaPredicted

3.1 / 100

WeirdMLPredicted

13.9 / 100

BigCodeBench Completebigcodebench

38.3 / 100

BigCodeBench Instructbigcodebench

30.0 / 100

BullshitBenchbullshitbench

2.0 / 100

Lech Mazur Writingepoch_ai

6.9

OTIS Mock AIME 2024–2025epoch_ai

0.1

LongBench v2 Easylongbench2

33.6

LongBench v2 Hardlongbench2

29.7

LongBench v2longbench2

26.6 / 100

NoLiMa (NIAH)nolima

87.9 / 100

WildBenchwildbench

55.8