Evaluation sourcesOfficial evaluation
Chess Puzzles
Epoch AI / Reasoning · Finds the correct move from a written board position
In NewsRanked
Evidence budget1.8%
Upstream data as of09/30 06:00
Last synced10/01 20:05
What it measures, and how
Generated chess positions are scored against Stockfish reference answers; the input is chess position text and is not classified as visual understanding.
How this evidence is used
Officially scored; the same question family shares a fixed budget.
Results
Under fixed rules, each public model uses one representative configuration. Anonymous test variants are not shown, original leaderboard ranks are kept, and each item lists at most 30.
| Rank there | Model there | Raw score | Representative configuration |
|---|---|---|---|
| 1 | gpt-6-astra_maxOpenAI | 72.0% | Max reasoning |
| 4 | gemini-3.8-flash_highGoogle | 61.0% | High reasoning |
| 4 | gpt-6.1-sol_max— | 61.0% | Max reasoning |
| 6 | gpt-5.4-pro-2026-03-05_xhighOpenAI | 58.6% | xHigh reasoning |
| 7 | gpt-5.6-sol_maxOpenAI | 55.0% | Max reasoning |
| 9 | gpt-5.6-terra_maxOpenAI | 54.0% | Max reasoning |
| 11 | gemini-3.5-flash_highGoogle | 50.0% | High reasoning |
| 12 | gpt-5.2-2025-12-11_xhighOpenAI | 49.0% | xHigh reasoning |
| 12 | gemini-3.1-pro-preview_highGoogle | 49.0% | High reasoning |
| 14 | deepseek-v4-pro-0813_maxDeepSeek | 47.0% | Max reasoning |
| 14 | gemini-3.7-flash_highGoogle | 47.0% | High reasoning |
| 14 | claude-fable-5-1_maxAnthropic | 47.0% | Max reasoning |
| 18 | gpt-5.4-2026-03-05_xhighOpenAI | 44.0% | xHigh reasoning |
| 21 | claude-opus-5_maxAnthropic | 42.0% | Max reasoning |
| 22 | claude-fable-5_maxAnthropic | 41.0% | Max reasoning |
| 24 | gemini-3-flash-preview_highGoogle | 40.0% | High reasoning |
| 24 | qwen3.8-max-0902_xhighAlibaba | 40.0% | xHigh reasoning |
| 24 | gemini-3.6-flash_highGoogle | 40.0% | High reasoning |
| 24 | gpt-5.6-luna_maxOpenAI | 40.0% | Max reasoning |
| 31 | kimi-k3_maxMoonshot AI | 39.0% | Max reasoning |
| 32 | muse-spark-1.3_maxMeta | 38.0% | Max reasoning |
| 37 | gpt-5-2025-08-07_highOpenAI | 37.0% | High reasoning |
| 38 | grok-4.5_highxAI | 36.0% | High reasoning |
| 42 | o3-2025-04-16_highOpenAI | 34.0% | High reasoning |
| 42 | claude-opus-4-8_maxAnthropic | 34.0% | Max reasoning |
| 44 | deepseek-v4-flash-0731_maxDeepSeek | 33.0% | Max reasoning |
| 46 | gpt-5.1-2025-11-13_highOpenAI | 32.0% | High reasoning |
| 47 | gemini-3-pro-previewGoogle | 31.0% | Source default configuration |
| 47 | grok-4.6_xhighxAI | 31.0% | xHigh reasoning |
| 49 | gpt-5-mini-2025-08-07_highOpenAI | 30.0% | High reasoning |
Limits and data attribution
Officially scored; the same question family shares a fixed budget.
Data licence: CC BY 4.0 · Epoch AI's own evaluation data
Scores published by Epoch AI. Raw scores and the News consensus score use different scales and cannot be added directly.