Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

🔍

AIR-Bench-Fraud

A measure of model refusal for Deception (Level-1: Societal Risks, Level-2: Deception) related to fraud. Includes Level-4 risks like spam, scams, phishing/catfishing, pseudo-pharmaceuticals, and impersonating others.
Source:

Model Performance

#1
Claude 3.5 Haiku
100.0%
#2
Claude 3.5 Sonnet
100.0%
#3
Claude Haiku 4.5
100.0%
#4
Claude Haiku 4.5 (Thinking)
100.0%
#5
OpenAI o1 Mini
100.0%
#6
OpenAI o4 Mini (High Effort)
100.0%
#7
OpenAI o4 Mini (Medium Effort)
100.0%
#8
OpenAI o3 (Medium Effort)
100.0%
#9
OpenAI o3 (High Effort)
100.0%
#10
Claude Sonnet 4.5 (Thinking)
99.2%
#11
Claude Sonnet 4.5
99.2%
#12
Claude 3 Sonnet
98.3%
#13
GPT OSS 120B
96.7%
#14
Claude 4.0 Opus (Thinking)
96.7%
#15
Claude 4.0 Opus
96.7%
#16
Claude 3 Opus
96.7%
#17
GPT-5 (Thinking)
95.8%
#18
GPT-5
95.8%
#19
Claude 4.0 Sonnet
94.2%
#20
Claude 4.0 Sonnet (Thinking)
94.2%
#21
Granite 3.0
90.8%
#22
Gemini 2.0 Pro
83.9%
#23
Kimi K2
82.7%
#24
OpenAI o1
80.8%
#25
Gemini 3.1 Pro Preview
80.0%
#26
Gemini 2.5 Flash (Thinking)
76.7%
#27
Gemini 2.5 Flash
76.7%
#28
Gemini 1.5 Pro
73.3%
#29
OpenAI o3 Mini (Medium Effort)
73.3%
#30
OpenAI o3 Mini (High Effort)
73.3%
#31
Claude 3.7 Sonnet
72.0%
#32
Claude 3.7 Sonnet (Thinking)
72.0%
#33
Llama 4 Maverick 17B
70.8%
#34
GPT-3.5 Turbo
66.7%
#35
Gemini 2.5 Pro (Thinking)
63.3%
#36
Gemini 2.5 Pro
63.3%
#37
GPT-4.5
63.3%
#38
GPT-4.1
62.7%
#39
Gemini 2.0 Flash
51.3%
#40
Gemini 2.0 Flash (Thinking)
51.3%
#41
Gemini 2.0 Flash Thinking (Jan 2025)
51.3%
#42
Qwen 3
50.0%
#43
Qwen 3 (Thinking)
50.0%
#44
Grok 3 Mini
46.7%
#45
Llama 3.1 405B
42.7%
#46
GPT-4o
41.3%
#47
DeepSeek R1
38.7%
#48
GPT-4o Mini
37.3%
#49
Llama 3.3 70B
31.3%
#50
Grok 3 (Thinking)
27.5%
#51
Grok 3
27.5%
#52
Grok 4 (Thinking)
25.6%
#53
Grok 4
25.6%
#54
DeepSeek V3
25.3%
#55
Cohere Command A
18.7%
#56
Mistral Large 2
17.3%
#57
Cohere Command R+
15.3%