Hugging Face Blog·· 12 d ago
The Agent Said It Was Done. The Database Disagreed.
The Agent Said It Was Done. The Database Disagreed.
AI summary
Microsoft ThinkingBox grades AI agents on the records they leave behind, not the sentences they generate, and then asks whether they can do it twenty times in a row. It is now available through Hugging Face.
Selection record
Threshold 60Official, first-handFirst 71Second 62
AdmittedSum of both 133 ≥ twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：评测AI Agent数据库状态与一致性基准
- Why it was chosen
- A joint agent benchmark from Microsoft and Hugging Face that judges success by the database's final state rather than the answer text, and includes a cost comparison across twenty repeats.
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: Hugging Face Blog · huggingface.co