Evaluation sourcesOfficial evaluation
AutomationBench
Zapier / Professional work · Automating work across office software
In NewsObserving
Evidence budgetNot scored
Upstream data as ofTo be confirmed
Last syncedNot collected yet
What it measures, and how
Only configurations on the same official private test set are compared; the Fable 5.1 fallback row that mixes in Opus 5 must be excluded.
How this evidence is used
Under observation: a public problem set is not the same as official private leaderboard data; stable exports and mixed-model labelling still need confirmation.
Limits and data attribution
Under observation: a public problem set is not the same as official private leaderboard data; stable exports and mixed-model labelling still need confirmation.
Data licence: Leaderboard data usage boundaries to be confirmed
Scores published by Zapier. Raw scores and the News consensus score use different scales and cannot be added directly.