Evaluation sourcesOfficial evaluation
Mystery Game Puzzles
Epoch AI / Reasoning · Deriving correct actions from unfamiliar rules
In NewsRanked
Evidence budget1.2%
Upstream data as of09/30 06:00
Last synced10/01 20:05
What it measures, and how
Only the mean_score of the Epoch standard protocol is used; the restriction that questions are not public is explicitly retained.
How this evidence is used
Officially scored; the same question family shares a fixed budget.
Results
Under fixed rules, each public model uses one representative configuration. Anonymous test variants are not shown, original leaderboard ranks are kept, and each item lists at most 30.
| Rank there | Model there | Raw score | Representative configuration |
|---|---|---|---|
| 1 | gpt-6-astra_maxOpenAI | 84.0% | Max reasoning |
| 2 | gpt-6.1-sol_max— | 80.0% | Max reasoning |
| 3 | claude-opus-5-5_maxAnthropic | 71.0% | Max reasoning |
| 4 | claude-sonnet-5-5_maxanthropic | 65.0% | Max reasoning |
| 5 | claude-opus-5_maxAnthropic | 59.0% | Max reasoning |
| 6 | gpt-5.6-sol_maxOpenAI | 58.0% | Max reasoning |
| 6 | claude-fable-5-1_maxAnthropic | 58.0% | Max reasoning |
| 8 | gpt-5.5_xhighOpenAI | 56.0% | xHigh reasoning |
| 8 | gpt-6-sol_maxOpenAI | 56.0% | Max reasoning |
| 10 | claude-fable-5_maxAnthropic | 52.0% | Max reasoning |
| 12 | gemini-3.8-flash_highGoogle | 47.0% | High reasoning |
| 13 | deepseek-v4-pro-0813_maxDeepSeek | 43.0% | Max reasoning |
| 14 | qwen3.8-max_xhighAlibaba | 38.0% | xHigh reasoning |
| 15 | gemini-3.7-flash_highGoogle | 37.0% | High reasoning |
| 15 | gpt-5.4-2026-03-05_xhighOpenAI | 37.0% | xHigh reasoning |
| 18 | claude-opus-4-8_maxAnthropic | 36.0% | Max reasoning |
| 19 | claude-sonnet-5_maxAnthropic | 35.0% | Max reasoning |
| 19 | gpt-5.6-terra_maxOpenAI | 35.0% | Max reasoning |
| 21 | deepseek-v4-flash-0731_maxDeepSeek | 34.0% | Max reasoning |
| 21 | gemini-3.1-pro-preview_highGoogle | 34.0% | High reasoning |
| 21 | grok-4.6_xhighxAI | 34.0% | xHigh reasoning |
| 24 | glm-5.3_maxZ.ai | 33.0% | Max reasoning |
| 26 | qwen3.7-maxAlibaba | 32.0% | Source default configuration |
| 26 | gemini-3.5-flash_highGoogle | 32.0% | High reasoning |
| 30 | gemini-3.6-flash_highGoogle | 30.0% | High reasoning |
| 31 | o3-2025-04-16_highOpenAI | 29.0% | High reasoning |
| 33 | claude-opus-4-7_maxAnthropic | 28.0% | Max reasoning |
| 37 | kimi-k3_maxMoonshot AI | 26.0% | Max reasoning |
| 41 | claude-opus-4-6_maxAnthropic | 25.0% | Max reasoning |
| 41 | muse-spark-1.3_maxMeta | 25.0% | Max reasoning |
Limits and data attribution
Officially scored; the same question family shares a fixed budget.
Data licence: CC BY 4.0 · Epoch AI's own evaluation data
Scores published by Epoch AI. Raw scores and the News consensus score use different scales and cannot be added directly.