The Decoder· Jonathan Kemper·· 12 d ago
Google researchers find a way to keep self-improving AI agents from memorizing their tests
Google researchers find a way to keep self-improving AI agents from memorizing their tests
AI summary
Self-improving AI agents tend to memorize their test tasks, so their gains shrink or disappear on new ones. RRSI, a new method from Google researchers, reins in this effect and lifts scores on unseen benchmarks by up to 4.7 points while using about 30 percent fewer tokens than an unregularized version.
Selection record
Threshold 76Media and individualsFirst 62Second 62
Not admittedSum of both 124 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：研究AI智能体自我改进与测试记忆
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: The Decoder · the-decoder.com