The Decoder· Jonathan Kemper·· 4 d ago
Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots
Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots
AI summary
Sakana AI's MASS method lets one language model improve itself by solving tasks, judging results, and refining workflows, reaching a win rate of nearly 70 percent against the base model after two rounds. But when the model misses the same errors across all three roles, it can mistake them for progress and reinforce them during training.
Selection record
Threshold 76Media and individualsFirst 62Second 62
Not admittedSum of both 124 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：Sakana AI自改进模型与多智能体研究
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: The Decoder · the-decoder.com