The Decoder· Jonathan Kemper·· 8 d ago
AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test
AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test
AI summary
Alibaba's new coding-agent benchmark, Co-Design Bench, tests AI agents on real user requests that already tripped them up in practice. Even GPT-6 Astra, the top performer, fully solves only 26.4 percent of tasks. On older benchmarks, the same models score above 99 percent.
Selection record
Threshold 76Media and individualsFirst 72Second 62
Not admittedSum of both 134 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：AI编码智能体基准测试与表现分析
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: The Decoder · the-decoder.com