The Decoder· Jonathan Kemper·· 6 h ago
Fancy harnesses make coding agents more expensive but not better, study finds
Fancy harnesses make coding agents more expensive but not better, study finds
AI summary
A controlled study shows that switching between coding-agent harnesses like Claude Code, OpenCode, or mini-SWE-agent barely moves pass rates more than rerunning the same setup. Cost per task, however, can triple, driven mainly by the system prompt that gets resent with every step.
Selection record
Threshold 76Media and individualsFirst 58Second 58
Not admittedSum of both 116 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：研究编码Agent框架对成本与效果的影响
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: The Decoder · the-decoder.com