Benchmark Details - Model Trust Scores

Benchmarks

🔍

Capability & Safety Benchmarks

MedQA

Speed & Latency Metrics

Median Tokens/s

AIR-Bench-AutonomousUnsafeOperations

A measure of model refusal for Operational Misuses (Level-1: System and Operational Risks, Level-2: Operational Misuses) related to autonomous unsafe operation of systems. Includes Level-4 risks like heavy machinery, transportation, energy/electrical grids, nuclear facilities, aircraft navigation/air traffic control, communication systems, water treatment facilities, life support, weapon systems/battlefield management, emergency services, and other unauthorized actions on behalf of users.

Model Performance

Claude Haiku 4.5

100.0%

Claude Haiku 4.5 (Thinking)

100.0%

Grok 4 (Thinking)

100.0%

Grok 4

100.0%

GPT OSS 120B

100.0%

Granite 3.0

90.3%

Claude 3.5 Sonnet

80.9%

OpenAI o3 (Medium Effort)

80.0%

OpenAI o3 (High Effort)

80.0%

#10

Claude Sonnet 4.5 (Thinking)

74.5%

#11

Claude Sonnet 4.5

74.5%

#12

Claude 4.0 Sonnet

73.0%

#13

Claude 4.0 Sonnet (Thinking)

73.0%

#14

Claude 3 Opus

72.4%

#15

Claude 4.0 Opus

70.7%

#16

Claude 4.0 Opus (Thinking)

70.7%

#17

Claude 3.5 Haiku

69.0%

#18

OpenAI o3 Mini (Medium Effort)

67.3%

#19

OpenAI o3 Mini (High Effort)

67.3%

#20

Claude 3 Sonnet

66.7%

#21

Gemini 3.1 Pro Preview

60.3%

#22

Gemini 1.5 Pro

58.5%

#23

GPT-5

58.0%

#24

GPT-5 (Thinking)

58.0%

#25

Claude 3.7 Sonnet

57.3%

#26

Claude 3.7 Sonnet (Thinking)

57.3%

#27

Llama 4 Maverick 17B

53.9%

#28

Gemini 2.0 Flash

52.7%

#29

Gemini 2.0 Flash Thinking (Jan 2025)

52.7%

#30

Gemini 2.0 Flash (Thinking)

52.7%

#31

Gemini 2.5 Flash

52.4%

#32

Gemini 2.5 Flash (Thinking)

52.4%

#33

GPT-4.5

49.4%

#34

Llama 3.1 405B

48.2%

#35

Kimi K2

46.7%

#36

GPT-4o Mini

39.4%

#37

Gemini 2.5 Pro (Thinking)

38.3%

#38

Gemini 2.5 Pro

38.3%

#39

OpenAI o1 Mini

35.8%

#40

Qwen 3

34.8%

#41

Qwen 3 (Thinking)

34.8%

#42

GPT-4o

33.3%

#43

OpenAI o1

33.0%

#44

OpenAI o4 Mini (Medium Effort)

30.0%

#45

OpenAI o4 Mini (High Effort)

30.0%

#46

Gemini 2.0 Pro

30.0%

#47

GPT-3.5 Turbo

30.0%

#48

Llama 3.3 70B

27.0%

#49

DeepSeek R1

24.8%

#50

DeepSeek V3

21.2%

#51

Mistral Large 2

20.6%

#52

Cohere Command A

19.7%

#53

GPT-4.1

18.3%

#54

Grok 3 (Thinking)

16.7%

#55

Grok 3

16.7%

#56

Grok 3 Mini

16.3%

#57

Cohere Command R+

15.2%

Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

AIR-Bench-AutonomousUnsafeOperations

Model Performance