Detailed results
100 repositories · 4.1k bugs · Updated September 24, 2026| Rank | Model | Agent | Bugs resolvedScore | Total USDUSD | USD / repo | Turns / repo | Tokens / repo | |
|---|---|---|---|---|---|---|---|---|
| 1 |  |
Sol 5.6 (xhigh)OpenAI | mini-SWE-agent | 4.7% | $7,230 | $72.30 | 233 | 104.8k |
| 2 | |
Luna 5.6 (xhigh)OpenAI | mini-SWE-agent | 2.5% | $224 | $2.24 | 204 | 61.2k |
| 3 | |
Terra 5.6 (xhigh)OpenAI | mini-SWE-agent | 1.5% | $357 | $3.57 | 76 | 46.6k |
| 4 | |
Luna 5.6 (high)OpenAI | mini-SWE-agent | 1.4% | $28 | $0.28 | 75 | 20.3k |
| 5 |  |
Opus 5 (xhigh)Anthropic | mini-SWE-agent | 1.3% | $5,363 | $53.63 | 323 | 192.7k |
| 6 |  |
Kimi K3Moonshot AI | mini-SWE-agent | 0.6% | $2,451 | $24.51 | 337 | 152.6k |
| 7 | |
Luna 5.6OpenAI | mini-SWE-agent | 0.5% | $4 | $0.04 | 22 | 4.5k |
| 8 | |
GPT-5.4 Mini (high)OpenAI | mini-SWE-agent | 0.5% | $122 | $1.22 | 75 | 40.5k |
| 9 | |
GPT-5.4 MiniOpenAI | mini-SWE-agent | 0.2% | $5 | $0.05 | 13 | 2.1k |
| 10 |  |
Gemini 3.5 Flash LiteGoogle | mini-SWE-agent | 0.1% | $6 | $0.06 | 34 | 5.2k |
Per-repository resource use includes non-deprecated retries and is averaged over evaluated repositories.
Pareto frontier
Hover a point for details · The line marks the Pareto frontier (best result per cost) · Click a point to see model details
Bugs resolved by model × repository
All models × 100 repositories
0%
100%
Hover for details · Click to open task