Detailed results

100 repositories · 4.1k bugs · Updated November 2, 2026
RankModelAgent Bugs resolvedScore USD / repo Turns / repo Tokens / repo
1 Astra 6OpenAI mini-SWE-agent 9.5% $35.00 113 119.5k
2 Sol 5.6 (xhigh)OpenAI mini-SWE-agent 4.7% $72.30 233 104.8k
3 Opus 5.5 (xhigh)Anthropic mini-SWE-agent 3.6% $21.98 266 112.7k
4 Fable 5.1Anthropic mini-SWE-agent 2.9% $39.04 319 165.0k
5 Luna 5.6 (xhigh)OpenAI mini-SWE-agent 2.5% $2.24 204 61.2k
6 DeepSeek V4.1 Flash (max)DeepSeek mini-SWE-agent 1.7% n/a n/a
7 Terra 5.6 (xhigh)OpenAI mini-SWE-agent 1.5% $3.57 76 46.6k
8 Luna 5.6 (high)OpenAI mini-SWE-agent 1.4% $0.28 75 20.3k
9 Opus 5 (xhigh)Anthropic mini-SWE-agent 1.3% $53.63 323 192.7k
10 DeepSeek V4.1 Flash (high)DeepSeek mini-SWE-agent 0.9% $0.97 347 241.1k
11 GLM 5.3 Flash (max)Zhipu AI mini-SWE-agent 0.7% n/a n/a
12 MiniMax M3.1 Flash (max)MiniMax mini-SWE-agent 0.7% $1.56 363 204.8k
13 GLM 5.3 Flash (high)Zhipu AI mini-SWE-agent 0.6% $1.81 332 115.6k
14 Kimi K3Moonshot AI mini-SWE-agent 0.6% $24.51 337 152.6k
15 Luna 5.6OpenAI mini-SWE-agent 0.5% $0.04 22 4.5k
16 GPT-5.4 Mini (high)OpenAI mini-SWE-agent 0.5% $1.22 75 40.5k
17 GPT-5.4 MiniOpenAI mini-SWE-agent 0.2% $0.05 13 2.1k
18 Gemini 3.5 Flash LiteGoogle mini-SWE-agent 0.1% $0.06 34 5.2k

Per-repository resource use includes non-deprecated retries and is averaged over evaluated repositories.

Hover a point for details · The line marks the Pareto frontier (best result per cost) · Click a point to see model details

All models × 100 repositories
0%
100%

Hover for details · Click to open task