Evaluation sourcesOfficial evaluation
Mystery Game Puzzles
Epoch AI / Reasoning · Deriving correct actions from unfamiliar rules
In NewsRanked
Evidence budget1.2%
Upstream data as of11/06 09:14
Last synced11/16 02:05
What it measures, and how
Only the mean_score of the Epoch standard protocol is used; the restriction that questions are not public is explicitly retained.
How this evidence is used
Officially scored; the same question family shares a fixed budget.
Results
Under fixed rules, each public model uses one representative configuration. Anonymous test variants are not shown, original leaderboard ranks are kept, and each item lists at most 30.
| Rank there | Model there | Raw score | Representative configuration |
|---|---|---|---|
| 1 | claude-fable-5-2_maxAnthropic | 90.0% | Max reasoning |
| 2 | gpt-6-astra_maxOpenAI | 84.0% | Max reasoning |
| 3 | gpt-6.1-sol_max— | 80.0% | Max reasoning |
| 4 | gpt-6.1-terra_maxOpenAI | 74.0% | Max reasoning |
| 5 | gemini-4-carbon_highgoogle | 73.0% | High reasoning |
| 6 | claude-opus-5-5_maxAnthropic | 71.0% | Max reasoning |
| 7 | claude-sonnet-5-5_maxanthropic | 65.0% | Max reasoning |
| 8 | gemini-4-argon_highGoogle | 64.0% | High reasoning |
| 9 | claude-opus-5_maxAnthropic | 59.0% | Max reasoning |
| 10 | gpt-5.6-sol_maxOpenAI | 58.0% | Max reasoning |
| 10 | claude-fable-5-1_maxAnthropic | 58.0% | Max reasoning |
| 12 | gpt-5.5_xhighOpenAI | 56.0% | xHigh reasoning |
| 12 | gpt-6-sol_maxOpenAI | 56.0% | Max reasoning |
| 14 | claude-fable-5_maxAnthropic | 52.0% | Max reasoning |
| 16 | muse-wonder-1.0_max— | 51.0% | Max reasoning |
| 17 | deepseek-v4.1-flash-1017_maxdeepseek | 49.0% | Max reasoning |
| 18 | gemini-3.8-flash_highGoogle | 47.0% | High reasoning |
| 19 | deepseek-v4-pro-0813_maxDeepSeek | 43.0% | Max reasoning |
| 19 | deepseek-v4.1-flash_maxDeepSeek | 43.0% | Max reasoning |
| 21 | kimi-k3.1-pro-together_maxMoonshot | 42.0% | Max reasoning |
| 22 | qwen3.8-max_xhighAlibaba | 38.0% | xHigh reasoning |
| 23 | gpt-5.4-2026-03-05_xhighOpenAI | 37.0% | xHigh reasoning |
| 23 | gemini-3.7-flash_highGoogle | 37.0% | High reasoning |
| 26 | claude-opus-4-8_maxAnthropic | 36.0% | Max reasoning |
| 27 | gpt-5.6-terra_maxOpenAI | 35.0% | Max reasoning |
| 27 | claude-sonnet-5_maxAnthropic | 35.0% | Max reasoning |
| 29 | grok-4.6_xhighxAI | 34.0% | xHigh reasoning |
| 29 | gemini-3.1-pro-preview_highGoogle | 34.0% | High reasoning |
| 29 | deepseek-v4-flash-0731_maxDeepSeek | 34.0% | Max reasoning |
| 32 | glm-5.3_maxZ.ai | 33.0% | Max reasoning |
Limits and data attribution
Officially scored; the same question family shares a fixed budget.
Data licence: CC BY 4.0 · Epoch AI's own evaluation data
Scores published by Epoch AI. Raw scores and the News consensus score use different scales and cannot be added directly.