Evaluation sourcesOfficial evaluation
MirrorCode
Epoch AI / Coding · Long-horizon software reproduction
In NewsReference only
Evidence budgetNot scored
Upstream data as of11/02 22:08
Last synced11/16 02:05
What it measures, and how
The published data covers only a few models and run times are very long; the sample and run conditions are insufficient to form a classification consensus.
How this evidence is used
For reference only; not counted towards overall or category scores.
Results
Under fixed rules, each public model uses one representative configuration. Anonymous test variants are not shown, original leaderboard ranks are kept, and each item lists at most 30.
| Rank there | Model there | Raw score | Representative configuration |
|---|---|---|---|
| 1 | claude-sonnet-5-5_maxanthropic | 97.8% | Max reasoning |
| 2 | claude-opus-5-5_maxAnthropic | 95.6% | Max reasoning |
| 3 | claude-fable-5-1_maxAnthropic | 93.3% | Max reasoning |
| 4 | claude-opus-5_maxAnthropic | 87.8% | Max reasoning |
| 6 | claude-haiku-5-5_max— | 78.9% | Max reasoning |
| 7 | gpt-6.1-terra_maxOpenAI | 76.7% | Max reasoning |
| 9 | claude-fable-5_maxAnthropic | 70.0% | Max reasoning |
| 10 | claude-opus-4-8_maxAnthropic | 65.6% | Max reasoning |
| 11 | deepseek-v4.1-flash_maxDeepSeek | 64.4% | Max reasoning |
| 13 | claude-sonnet-5_maxAnthropic | 63.3% | Max reasoning |
| 14 | gpt-6.1-sol_max— | 61.1% | Max reasoning |
| 15 | gpt-6-astra_maxOpenAI | 60.0% | Max reasoning |
| 16 | gemini-4-argon_highGoogle | 58.9% | High reasoning |
| 17 | grok-4.7_xhighxAI | 54.4% | xHigh reasoning |
| 17 | kimi-k3_maxMoonshot AI | 54.4% | Max reasoning |
| 17 | muse-spark-1.3_maxMeta | 54.4% | Max reasoning |
| 20 | qwen3.8-max_xhighAlibaba | 52.2% | xHigh reasoning |
| 21 | gpt-6-sol_maxOpenAI | 48.9% | Max reasoning |
| 22 | qwen3.8-max-0902_xhighAlibaba | 47.8% | xHigh reasoning |
| 24 | muse-wonder-1.0_max— | 45.6% | Max reasoning |
| 25 | claude-opus-4-7_maxAnthropic | 43.3% | Max reasoning |
| 26 | grok-4.6_xhighxAI | 41.1% | xHigh reasoning |
| 26 | gpt-5.6-terra_maxOpenAI | 41.1% | Max reasoning |
| 26 | gemini-3.7-flash_highGoogle | 41.1% | High reasoning |
| 29 | mistral-large-4-0_highMistral AI | 40.0% | High reasoning |
| 30 | gemini-3.8-flash_highGoogle | 38.9% | High reasoning |
| 31 | glm-5.2_maxZ.ai | 36.7% | Max reasoning |
| 32 | claude-opus-4-6_maxAnthropic | 33.3% | Max reasoning |
| 32 | deepseek-v4-pro-0813_maxDeepSeek | 33.3% | Max reasoning |
| 34 | grok-4.5_highxAI | 32.2% | High reasoning |
Limits and data attribution
For reference only; not counted towards overall or category scores.
Data licence: CC BY 4.0 · Epoch AI's own evaluation data
Scores published by Epoch AI. Raw scores and the News consensus score use different scales and cannot be added directly.