Gemma 3 27B Instruct

March 12, 2025Last synced Apr 7, 2026, 4:00 PM

Blended Price

Free/M

Input Price

Free/M

Output Price

Free/M

Speed

32 tok/s

TTFT

1.84s

Benchmark Scores

Intelligence Index

External Benchmarks

AIME 2024benchmark_matrix

How Gemma 3 27B Instruct Compares

Axes

X Axis

Y Axis

Bubble Size

Filters

Blended Price (USD)$0.00 – $30.0

AgMoBench Overall≥ 3.5

Providers

ai21-labsalibabaanthropicawsazurebaiducoheredeepseek

Show quadrants

anthropic

openai

google

meta

mistral

nvidia

kimi

xai

azure

deepseek

aws

cohere

baidu

ai21-labs

zai

reka-ai

xiaomi

alibaba

minimax

ibm

Bubble size = Context Window

16384.00

2000000.00

Compare with other models

GPQA Diamond

0.4 / 100

LiveCodeBench

0.1 / 100

Terminal-Bench Hard

0.0 / 100

τ²-Bench

0.1 / 100

Arena-Hard Autobenchmark_matrix

69.9

Chatbot Arena ELO (Matrix)benchmark_matrix

1338.0

GPQA Diamond (Matrix)benchmark_matrix

42.4

GSM8Kbenchmark_matrix

92.3

HumanEvalbenchmark_matrix

87.8

IFEvalbenchmark_matrix

78.0

LiveCodeBench (Matrix)benchmark_matrix

29.7

MATH-500 (Matrix)benchmark_matrix

78.0

MMLUbenchmark_matrix

76.9

MMLU-Pro (Matrix)benchmark_matrix

67.5

SimpleQAbenchmark_matrix

22.0

simpleqabenchmark_matrix

22.0

SWE-bench Verifiedbenchmark_matrix

32.0 / 100

AA-Omniscience AccuracyPredicted

40.5 / 100

AA-Omniscience Hallucination RatePredicted

94.3 / 100

Aider PolyglotPredicted

20.2 / 100

AlpacaEval 2.0 LCPredicted

42.1 / 100

AlpacaEval 2.0 RawPredicted

31.3 / 100

ARC-AGI-1Predicted

3.1 / 100

ARC-AGI-1 Cost per TaskPredicted

0.0

ARC-AGI-2Predicted

0.0 / 100

ARC-AGI-2 Cost per TaskPredicted

0.0

BFCL (Berkeley Function Calling)Predicted

34.3

BigCodeBench CompletePredicted

55.0 / 100

BigCodeBench InstructPredicted

45.1 / 100

AA Intelligence Index (Matrix)Predicted

60.9

AA Long Context Reasoning (Matrix)Predicted

70.1

AIME 2025 (Matrix)Predicted

30.6

BrowseCompPredicted

58.8

BRUMO 2025Predicted

75.7

CMIMC 2025Predicted

67.3

HLE (Matrix)Predicted

11.3

HMMT Feb 2025Predicted

71.4

HMMT Nov 2025Predicted

88.5

IFBench (Matrix)Predicted

24.3

IMO 2025Predicted

10.6

MathArena Apex 2025Predicted

0.7

MMMU-ProPredicted

80.2

MRCR v2Predicted

74.1

OSWorldPredicted

34.0

SMT 2025Predicted

76.2

SWE-bench ProPredicted

29.3

Tau-Bench Telecom (Matrix)Predicted

95.0

Terminal-Bench 2.0Predicted

18.6

Terminal-Bench 1.0Predicted

12.6

USAMO 2025Predicted

5.1

Video-MMUPredicted

78.2

browsecompPredicted

55.9

BullshitBenchPredicted

11.8 / 100

Aider PolyglotPredicted

0.0

Apex AgentsPredicted

1.7

Arc Agi 2Predicted

0.0

BIG-Bench HardPredicted

3.0

CAD-EvalPredicted

6.7

Chess PuzzlesPredicted

0.1

CyBenchPredicted

0.1

DeepResearchBenchPredicted

0.3

FictionLiveBenchPredicted

0.3

GeoBenchPredicted

0.0

GSM8K (Epoch)Predicted

12.1

HellaSwagPredicted

19.3

Lech Mazur WritingPredicted

7.0

METR Time HorizonsPredicted

0.3

OTIS Mock AIME 2024–2025Predicted

0.1

PosttrainbenchPredicted

0.0

SimpleQA Verified (Epoch)Predicted

0.2

The Agent CompanyPredicted

1.2

TriviaQAPredicted

2.6

WinoGrandePredicted

0.9

FrontierMathPredicted

11.3 / 100

GAIA Level 1Predicted

30.3

GAIA Level 2Predicted

11.3

GAIA Level 3Predicted

0.1

GAIAPredicted

19.1 / 100

LegalBenchPredicted

82.1 / 100

LiveBench CodingPredicted

64.0 / 100

LiveBench Data AnalysisPredicted

38.8 / 100

LiveBench LanguagePredicted

47.9 / 100

LiveBench MathPredicted

56.2 / 100

LiveBench OverallPredicted

41.0 / 100

LiveBench ReasoningPredicted

33.5 / 100

LongBench v2 EasyPredicted

41.8

LongBench v2 HardPredicted

37.8

LongBench v2Predicted

35.9 / 100

MathVistaPredicted

57.5 / 100

MedQA (USMLE)Predicted

64.5

MLE-benchPredicted

15.7 / 100

MMTU Table UnderstandingPredicted

43.8 / 100

MT-BenchPredicted

7.9 / 10

OCRBench v2Predicted

78.6 / 100

Open LLM AveragePredicted

40.5 / 100

Open LLM: BBHPredicted

66.1 / 100

Open LLM: GPQAPredicted

34.6 / 100

Open LLM: IFEvalPredicted

73.7 / 100

Open LLM: MATH Level 5Predicted

41.0 / 100

Open LLM: MMLU-PROPredicted

51.4 / 100

Open LLM: MUSRPredicted

44.4 / 100

RE-BenchPredicted

0.3

SimpleBenchPredicted

20.7 / 100

SWE-bench LitePredicted

13.2 / 100

tau-bench RetailPredicted

67.6 / 100

Vectara Factual ConsistencyPredicted

93.8 / 100

Vectara Hallucination RatePredicted

6.2 / 100

WebArenaPredicted

5.9 / 100

WeirdMLPredicted

25.3 / 100

WildBenchPredicted

50.5

Epoch Capabilities Indexepoch_ai

131.1

NoLiMa (NIAH)nolima

88.6 / 100