Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

🔍

SWE-bench

Evaluates the model's ability to resolve real-world software engineering issues from GitHub repositories. Models are tested on their capacity to generate code patches that fix actual bugs and implement requested features from open-source projects.
Source:

Model Performance

#1
Claude Opus 4.6 (Thinking)
78.2%
#2
Claude Sonnet 4.6
77.4%
#3
Gemini 3.1 Pro Preview
76.4%
#4
Gemini 3.0 Flash
75.0%
#5
GPT-5.4 Mini
73.0%
#6
GPT-5.1 Codex Max
71.0%
#7
Claude Sonnet 4.5 (Thinking)
70.0%
#8
GPT-5.4 Nano
69.8%
#9
GPT-5
69.0%
#10
GPT-5 (Thinking)
69.0%
#11
DeepSeek V3.2
68.4%
#12
DeepSeek V3.2 (Thinking)
67.2%
#13
Claude Haiku 4.5 (Thinking)
66.6%
#14
Claude 4.0 Sonnet
65.0%
#15
Grok 4
58.6%
#16
Grok 4 (Thinking)
58.6%
#17
OpenAI o3 (Medium Effort)
49.8%
#18
OpenAI o3 (High Effort)
49.8%
#19
GPT-4.1
47.4%
#20
Qwen 3 Max (Thinking)
44.4%
#21
Qwen 3 Max Preview
44.4%
#22
Grok 3
42.0%
#23
Grok 3 (Thinking)
42.0%
#24
Kimi K2
34.2%
#25
OpenAI o4 Mini (High Effort)
33.4%
#26
OpenAI o4 Mini (Medium Effort)
33.4%
#27
GPT OSS 120B
25.2%
#28
Llama 4 Maverick 17B
18.4%
#29
Claude Opus 4.5
15.9%
#30
Grok 4.20
8.8%
#31
Cohere Command A
0.2%
#32
Claude Opus 4.5 (Thinking)
0.0%
#33
GPT-5.2
0.0%
#34
Grok 4.20 (Reasoning)
0.0%