Microsoft researcher discusses AI evaluation and unexpected model failures
What happened
On 7 October 2026, Microsoft Research released a podcast episode in which Microsoft researcher Jennifer Neville discussed AI evaluation. She spoke about the role of evaluation in pushing the performance boundaries of AI systems, and about the "unexpected failures" that occur when models are tested outside traditional benchmarks. Drawing on her own path from mathematics and physics to cognitive science and computer science, she also shared practical advice for using current AI systems, and stressed that data should be examined carefully when results exceed expectations.
Written by AI from the reports · updated 10 d ago
Timeline
Follow the reports to see the event from each side.
- Microsoft ResearchWhat AI gets wrong and what failure teaches us
Jennifer Neville did not want to go into computer science—but that’s exactly where she landed. Neville discusses the starts and stops that led to her professional sweet spot and her work identifying “surprising failures” making it hard for AI to handle complexity.