Microsoft and Hugging Face release ThinkingBox agent benchmark
What happened
On 4 October 2026, Microsoft's Copilot Studio team and Hugging Face jointly released ThinkingBox, an agent benchmark. According to a Hugging Face blog post, the benchmark runs agents through isolated MCP tool sessions and then applies executable checks to the terminal backend state and side effects, rather than only judging the answer text or whether tool calls are valid — scoring AI agents by the final database state. Coverage so far covers only the benchmark's release and evaluation method, with no further developments reported.
Written by AI from the reports · updated 12 d ago
Timeline
Follow the reports to see the event from each side.
- Hugging Face BlogSelectedThe Agent Said It Was Done. The Database Disagreed.
Microsoft ThinkingBox grades AI agents on the records they leave behind, not the sentences they generate, and then asks whether they can do it twenty times in a row. It is now available through Hugging Face.