Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

🔍

AIR-Bench-PerpetuatingHarmfulStereotypes

A measure of model refusal for Hate/Toxicity (Level-1: Content Safety Risks, Level-2: Hate/Toxicity) related to perpetuating harmful beliefs. Includes Level-4 risks like negative stereotyping of any group, perpetuating racism, and perpetuating sexism.

Model Performance

#1
Claude 3 Opus
100.0%
#2
Claude 3 Sonnet
100.0%
#3
Claude 3.5 Haiku
100.0%
#4
Claude 4.0 Sonnet
100.0%
#5
Claude 4.0 Sonnet (Thinking)
100.0%
#6
GPT-5
100.0%
#7
GPT-5 (Thinking)
100.0%
#8
GPT-4.5
100.0%
#9
Gemini 3.1 Pro Preview
100.0%
#10
Claude 4.0 Opus (Thinking)
99.7%
#11
Claude 4.0 Opus
99.7%
#12
Claude Haiku 4.5
97.8%
#13
Claude Haiku 4.5 (Thinking)
97.8%
#14
Gemini 2.5 Pro
96.4%
#15
Gemini 2.5 Pro (Thinking)
96.4%
#16
OpenAI o4 Mini (High Effort)
95.6%
#17
OpenAI o4 Mini (Medium Effort)
95.6%
#18
GPT OSS 120B
95.4%
#19
Claude 3.5 Sonnet
95.2%
#20
Gemini 2.5 Flash (Thinking)
94.3%
#21
Gemini 2.5 Flash
94.3%
#22
OpenAI o3 Mini (High Effort)
93.9%
#23
OpenAI o3 Mini (Medium Effort)
93.9%
#24
OpenAI o1
90.7%
#25
Gemini 1.5 Pro
90.6%
#26
Granite 3.0
90.3%
#27
Llama 4 Maverick 17B
90.0%
#28
Claude Sonnet 4.5
88.9%
#29
Claude Sonnet 4.5 (Thinking)
88.9%
#30
Qwen 3
86.5%
#31
Qwen 3 (Thinking)
86.5%
#32
Claude 3.7 Sonnet
80.2%
#33
Claude 3.7 Sonnet (Thinking)
80.2%
#34
Gemini 2.0 Pro
80.0%
#35
Kimi K2
77.8%
#36
OpenAI o3 (Medium Effort)
70.7%
#37
OpenAI o3 (High Effort)
70.7%
#38
GPT-4.1
69.0%
#39
Gemini 2.0 Flash (Thinking)
65.1%
#40
Gemini 2.0 Flash Thinking (Jan 2025)
65.1%
#41
Gemini 2.0 Flash
65.1%
#42
DeepSeek R1
63.5%
#43
OpenAI o1 Mini
57.5%
#44
Llama 3.1 405B
53.2%
#45
GPT-4o
47.6%
#46
DeepSeek V3
46.0%
#47
GPT-4o Mini
44.4%
#48
Llama 3.3 70B
42.9%
#49
Grok 3 Mini
39.8%
#50
Grok 3 (Thinking)
35.5%
#51
Grok 3
35.5%
#52
Cohere Command A
33.3%
#53
Mistral Large 2
31.0%
#54
Cohere Command R+
22.2%
#55
Grok 4
13.3%
#56
Grok 4 (Thinking)
13.3%
#57
GPT-3.5 Turbo
10.0%