Hugging Face launched Open TTS Leaderboard to evaluate open-source TTS models using objective metrics, reducing evaluation from weeks of voting to hours. It uses Qwen3 ASR for WER/CER, measures RTFx and TTFA on H200 GPUs and calculates speaker similarity (SIM) with WavLM embeddings, supporting multilingual and voice-cloning comparisons.
Hugging Face published ProvenanceGuard, a post-generation verification layer for MCP agents that preserves tool-output provenance and detects cross-source confusion where a fact is true but attributed incorrectly. Across 281 real medical-agent traces, it blocked 138 of the 139 claims experts judged should be blocked. Source identification accuracy was around 86%, and it scored highest in comparisons with four fact-checkers.
The UK AI Security Institute (AISI) is publicly sharing evaluation results using EvalEval's Every Eval Ever schema and Evaluation Cards platform to improve reproducibility in evaluation science.
IBM researchers added consistency guidelines to ALTK-Evolve, using Consistency Analyzer to diagnose decision points in agent trajectories that are prone to flipping.
Hugging Face released BenchMIRT to audit LLM benchmarks at individual-prompt level using multidimensional item response theory (MIRT), separating dominant safety and general reasoning dimensions. Trained on 100 LLMs, 16 benchmarks and over 34K questions, it consistently recovered both without being told what each benchmark measured. Analysis finds safety benchmarks such as BBQ and WMDP correlate more with general reasoning, suggesting a single score can mix multiple signals.
Voice Arena and Hugging Face added Monsoon en-IN and Monsoon hi-IN evaluation datasets to Open ASR Leaderboard, making Hindi its first Indian language.
Multiverse Computing published a paper proposing Quantization-Aware Healing (QAH). After compressing GPT-OSS 120B to 60B parameters and quantising it to MXFP4, the method distils directly from the original uncompressed model rather than a reconstructed bfloat16 checkpoint.
Hugging Face proposed three tests to quantify benchmark optimisation in speech recognition: a consensus-disagreement probe, masked-entity retrieval and spelling switches. Evaluating 11 open-source ASR models, it found some high-scoring models reproduce errors in VoxPopuli and LibriSpeech reference transcripts even when contradicted by audio, when relevant words are muted or when both spellings fit the audio.
Hugging Face ran the ICML 2026 Open Reproduction Challenge from 15 July to 2 August. Using coding agents such as Claude Code, Codex and Cursor, 1,221 community members reproduced papers and published 6,816 Trackio logs covering 2,226 papers, around a third of the conference total.
Multiverse Computing published the paper Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss.
Hume released Real World VoiceEQ, a speech evaluation benchmark covering over 40 proprietary and open-source voice models, more than 15 evaluation dimensions and over 60 metrics across ASR, TTS, S2S and speech understanding.
Hugging Face released ScarfBench (Self-Contained Application Refactoring Benchmark) to evaluate AI agents migrating enterprise Java applications between Spring, Jakarta EE and Quarkus.
Hugging Face proposed DiScoFormer (Density and Score Transformer), estimating a distribution's density and score simultaneously in one forward pass from a set of data points, without retraining for new distributions.