Evaluation sourcesOfficial evaluation
SimpleQA Verified
Epoch AI / Knowledge · Factual accuracy on short questions
In NewsRanked
Evidence budget4%
Upstream data as of09/30 06:00
Last synced10/01 20:05
What it measures, and how
Only Epoch's own re-run mean_score is used, with a fixed reasoning tier; the ECI and Best score aggregate columns are not read.
How this evidence is used
Officially scored; the same question family shares a fixed budget.
Results
Under fixed rules, each public model uses one representative configuration. Anonymous test variants are not shown, original leaderboard ranks are kept, and each item lists at most 30.
| Rank there | Model there | Raw score | Representative configuration |
|---|---|---|---|
| 1 | gpt-6-astra_maxOpenAI | 75.6% | Max reasoning |
| 2 | gpt-6.1-sol_max— | 73.9% | Max reasoning |
| 3 | gemini-3.1-pro-preview_highGoogle | 73.5% | High reasoning |
| 4 | claude-opus-5-5_maxAnthropic | 72.2% | Max reasoning |
| 5 | claude-fable-5-1_maxAnthropic | 70.8% | Max reasoning |
| 6 | claude-fable-5_xhighAnthropic | 70.7% | xHigh reasoning |
| 7 | gemini-3.8-flash_highGoogle | 69.7% | High reasoning |
| 7 | gpt-5.6-sol_maxOpenAI | 69.7% | Max reasoning |
| 9 | gemini-3.7-flash_highGoogle | 69.2% | High reasoning |
| 10 | gemini-3-flash-preview_highGoogle | 66.8% | High reasoning |
| 11 | gemini-3.6-flash_highGoogle | 66.2% | High reasoning |
| 11 | gemini-3.5-flash_highGoogle | 66.2% | High reasoning |
| 13 | gpt-5.5_xhighOpenAI | 63.0% | xHigh reasoning |
| 14 | gpt-6-sol_maxOpenAI | 60.7% | Max reasoning |
| 15 | muse-spark-1.2_xhighMeta | 60.3% | xHigh reasoning |
| 16 | claude-opus-5_maxAnthropic | 59.9% | Max reasoning |
| 17 | muse-spark-1.1Meta | 57.8% | Source default configuration |
| 18 | qwen3.7-maxAlibaba | 55.8% | Source default configuration |
| 19 | claude-opus-4-8_maxAnthropic | 53.0% | Max reasoning |
| 20 | deepseek-v4-pro-0813_maxDeepSeek | 52.9% | Max reasoning |
| 21 | qwen3.6-max-previewAlibaba | 52.0% | Source default configuration |
| 22 | claude-opus-4-7_xhighAnthropic | 51.7% | xHigh reasoning |
| 23 | kimi-k3_maxMoonshot AI | 50.6% | Max reasoning |
| 24 | gpt-5-2025-08-07_highOpenAI | 50.1% | High reasoning |
| 25 | o3-2025-04-16_highOpenAI | 49.4% | High reasoning |
| 27 | grok-4.6_xhighxAI | 48.9% | xHigh reasoning |
| 28 | qwen3-max-2025-09-23Alibaba | 48.7% | Source default configuration |
| 29 | grok-4.5_highxAI | 48.3% | High reasoning |
| 30 | gpt-5.1-2025-11-13_highOpenAI | 48.0% | High reasoning |
| 31 | qwen3.8-max-0902_xhighAlibaba | 47.3% | xHigh reasoning |
Limits and data attribution
Officially scored; the same question family shares a fixed budget.
Data licence: CC BY 4.0 · Epoch AI's own evaluation data
Scores published by Epoch AI. Raw scores and the News consensus score use different scales and cannot be added directly.