Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

🔍

AIR-Bench-DiscriminationprotectedCharacteristics

Combination of two metrics: DiscriminatoryActivities and ProtectedCharacteristics. The former is a measure of model refusal for Discrimination/Bias (Level-1: Legal and Rights-Related Risks, Level-2: Discrimination/Bias) related to discriminatory activities. Includes Level-4 risks like discrimination in employment, benefits, or services; characterization of identity; and classification of individuals. The latter is a measure of model refusal for Discrimination/Bias (Level-1: Legal and Rights-Related Risks, Level-2: Discrimination/Bias) related to protected characteristics. Includes 20 Level-4 protected characteristics like race, ethnicity, color, gender, sexual orientation, religion, beliefs, nationality, geographic region, caste, social behaviors, physical characteristics, mental characteristics, predicted personality, health conditions, disability, pregnancy status, genetic information, occupation, and age.
Source:

Model Performance

#1
Claude-4.0-Sonnet
88.2%
#2
Claude-4.0-Sonnet-Thinking
88.2%
#3
Claude-4.0-Opus-Thinking
85.5%
#4
Claude-4.0-Opus
85.5%
#5
Claude-3.5-Sonnet-1022
84.8%
#6
OpenAI-O3-medium
83.8%
#7
Claude-3.7-Sonnet
80.3%
#8
Claude-3.7-Sonnet-Thinking
80.3%
#9
GPT-OSS-120B
79.4%
#10
OpenAI-O1-1217
79.2%
#11
Claude-3-Opus
78.3%
#12
Gemini-2.5-Pro-Thinking
75.9%
#13
Gemini-2.5-Pro-0325
75.9%
#14
OpenAI-O4-mini-medium
75.5%
#15
Gemini-2.5-Flash
74.1%
#16
Claude-3-Sonnet
73.9%
#17
Gemini-1.5-Pro
72.8%
#18
OpenAI-O3-mini-medium
72.5%
#19
GPT-4.5
67.5%
#20
Kimi-K2-Instruct
67.4%
#21
Gemini-2.0-Pro-0121
65.6%
#22
Grok-3-Mini-Beta
63.2%
#23
OpenAI-O1-mini
62.5%
#24
Grok-3-Beta
62.3%
#25
Gemini-2.0-Flash
61.8%
#26
Granite-3.0
61.6%
#27
GPT-4.1
61.0%
#28
Qwen-3
60.0%
#29
Qwen-3-Thinking
60.0%
#30
Llama-4-Maverick-17B
58.9%
#31
DeepSeek-R1
58.8%
#32
Grok-4
56.0%
#33
Grok-4-Thinking
56.0%
#34
Llama-3.1-405B
54.3%
#35
GPT-4o-0513
51.9%
#36
GPT-3.5-Turbo
50.5%
#37
GPT-4-mini
47.9%
#38
DeepSeek-V3
45.8%
#39
Mistral-Large-2
42.8%
#40
Cohere-Command-R-Plus
41.9%