Hugging Face Blog·· 6 h ago
StateBench: Testing how language models track state changes over long documents
StateBench: Testing how language models track state changes over long documents
AI summary
We’re introducing StateBench, a benchmark for long-context state tracking —how well language models keep track of information about entities in long documents as they evolve.
Selection record
Threshold 60Official, first-handFirst 38Second 42
Not admittedSum of both 80 < twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：语言模型状态追踪基准与架构评测
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Hugging Face Blog · huggingface.co