Benchmark Explorer

Explore how models perform on various benchmarks

Benchmarks

🔍

MedQA

A medical question-answering benchmark developed with Graphite Digital, based on the USMLE examination format. The evaluation includes two phases: an unbiased baseline assessment using 2,000 medical questions, and a bias injection phase that tests models' handling of racial bias in medical contexts. Questions cover graduate-level medical knowledge while also examining the impact of racial bias on model performance and medical decision-making.
Source:

Model Performance

#1
OpenAI-O1-1217
96.5%
#2
OpenAI-O3-high
96.1%
#3
OpenAI-O3-medium
96.1%
#4
OpenAI-O4-mini-high
96.0%
#5
OpenAI-O4-mini-medium
96.0%
#6
OpenAI-O3-mini-high
94.8%
#7
OpenAI-O3-mini-medium
94.8%
#8
Claude-4.1-Opus-Thinking
93.6%
#9
Gemini-2.5-Pro-Thinking
93.1%
#10
Gemini-2.5-Pro-0325
93.1%
#11
Claude-4.0-Opus
92.9%
#12
Claude-4.0-Sonnet-Thinking
92.7%
#13
Grok-4-Thinking
92.5%
#14
Grok-4
92.5%
#15
Claude-4.1-Opus
92.5%
#16
GPT-4.1
91.2%
#17
Gemini-2.5-Flash-Thinking
91.0%
#18
DeepSeek-R1
90.8%
#19
Qwen-3
90.6%
#20
Qwen-3-Thinking
90.6%
#21
Claude-4.0-Sonnet
90.3%
#22
OpenAI-O1-mini
90.2%
#23
Claude-3.7-Sonnet-Thinking
90.2%
#24
Grok-3-Mini-Beta
90.1%
#25
Llama-3.1-405B
88.2%
#26
Qwen-3-Max-Preview
87.4%
#27
Gemini-2.5-Flash
86.7%
#28
Kimi-K2-Instruct
84.0%
#29
Grok-3-Beta
83.9%
#30
Grok-3-Think
83.9%
#31
Claude-3.5-Sonnet-1022
83.2%
#32
DeepSeek-V3-0324
82.0%
#33
DeepSeek-V3
80.9%
#34
Cohere-Command-A
80.5%
#35
Mistral-Medium-3.1
78.2%
#36
Gemini-1.5-Pro
76.5%
#37
Mistral-Large-2
76.2%
#38
GPT-4-mini
72.4%
#39
GPT-3.5-Turbo
58.5%
#40
Cohere-Command-R-Plus
51.4%
#41
Llama-4-Maverick-17B
43.3%