The Decoder· Matthias Bastian·· 7 h ago
Rapid LLM development is outrunning the research meant to explain it, study finds
Rapid LLM development is outrunning the research meant to explain it, study finds
AI summary
Stanford researchers tested 50 widely cited claims about LLMs against 20 current models. 44 percent of the findings are already "expired," and another 34 percent have weakened significantly. The average lifespan is just 15.7 months. Claims about what models can't do expire fastest.
Selection record
Threshold 76Media and individualsFirst 58Second 61
Not admittedSum of both 119 < twice the threshold 152
- Source tier
- Media and individuals; this tier's threshold is 76
- Pre-filter
- passed：研究LLM能力结论时效性，实质AI评测
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: The Decoder · the-decoder.com