Google unveiled Gemini 4 Argon, its new frontier model that closes the gap with rivals from OpenAI and Anthropic, beating some of them on key benchmarks. While it may not clearly lead the pack, it is relatively cheap for a frontier model, at least at the introductory price.
Hugging Face launched Open TTS Leaderboard to evaluate open-source TTS models using objective metrics, reducing evaluation from weeks of voting to hours. It uses Qwen3 ASR for WER/CER, measures RTFx and TTFA on H200 GPUs and calculates speaker similarity (SIM) with WavLM embeddings, supporting multilingual and voice-cloning comparisons.
Anthropic released Claude Sonnet 5.5, officially claiming over 30% faster operation and up to 30% lower cost for most work. Pricing matches Sonnet 5, but it outperforms Sonnet 5 on every benchmark.
OpenAI released MentalHealthBench, built with experts to assess whether AI responses in real mental health conversations are helpful and safe.
Anthropic released Claude Opus 5.5, followed roughly an hour later by OpenAI’s GPT-6 Sol and GPT-6 Luna. Both GPT-6 Sol and Luna are priced at half their corresponding GPT-5.6 models.
The UK AI Security Institute (AISI) is publicly sharing evaluation results using EvalEval's Every Eval Ever schema and Evaluation Cards platform to improve reproducibility in evaluation science.
Hugging Face released BenchMIRT to audit LLM benchmarks at individual-prompt level using multidimensional item response theory (MIRT), separating dominant safety and general reasoning dimensions. Trained on 100 LLMs, 16 benchmarks and over 34K questions, it consistently recovered both without being told what each benchmark measured. Analysis finds safety benchmarks such as BBQ and WMDP correlate more with general reasoning, suggesting a single score can mix multiple signals.
Voice Arena and Hugging Face added Monsoon en-IN and Monsoon hi-IN evaluation datasets to Open ASR Leaderboard, making Hindi its first Indian language.
Google DeepMind announced the world’s first double-blind evaluation of proprietary frontier AI models, restricting external evaluations to cryptographically isolated environments to prevent models seeing test questions in advance. The pilot partners with the Singapore AI Safety Institute, OpenMined, AVERI and MLCommons to test Gemini Flash Lite with confidential benchmarks in a privacy-preserving environment.
METR research shows AI's effects on scientific discovery are uneven: cybersecurity vulnerability reporting accelerated sharply in 2026 compared with 2025, mathematics accelerated only modestly, and algorithmic progress in AI research itself showed no significant acceleration. A multi-university team proposed SPADE, alternating LLM generation of executable training environments with solving them. Qwen3-30B-A3B averaged 58.3 on the game-environment suite, 8.1 above baseline.
Hugging Face proposed three tests to quantify benchmark optimisation in speech recognition: a consensus-disagreement probe, masked-entity retrieval and spelling switches. Evaluating 11 open-source ASR models, it found some high-scoring models reproduce errors in VoxPopuli and LibriSpeech reference transcripts even when contradicted by audio, when relevant words are muted or when both spellings fit the audio.
Import AI 469 introduces DiG-bench (Discovery in Games), with 70 games whose hidden rules and goals must be discovered through interaction. Opus 5 and Fable 5 with Claude Code performed best; only they completed some Tier 7 tasks, while humans achieved 100% completion.
Microsoft Research introduced MindTopo to evaluate multimodal models' reasoning and planning over connectivity, enclosure, order, separation and knotting. Current models perform much better at static-image recognition than interactive planning, with failures mainly in planning rather than perception and overall performance far below humans. Image and video generation helps only when preserving structural relations in a single frame; multi-step operations often change topology or violate constraints.
Google Research's knowledge profiling framework evaluated 13 LLMs on WikiProfile's 2,150 Wikipedia facts. Gemini-3-Pro and GPT-5 encoded 95–98% but failed direct recall of 26–34%, and still failed 11–12% with thinking enabled. It argues frontier factual errors arise more from knowledge accessibility than absence, shifting the bottleneck from acquisition to use.
Microsoft Research released the open-source Orchard framework, centred on the Kubernetes environment service Orchard Env. It reuses environments, data pipelines and evaluation workflows across tasks and supports training agents directly within real deployment frameworks such as Codex, OpenClaw and ZeroClaw.
Epoch and METR released MirrorCode to test AI systems' ability to complete programming tasks requiring extended human effort. First announced in April, it is formally released after additional testing.
DharmaOCR scored 0.925 on a Portuguese OCR benchmark, ahead of Mistral OCR4's 0.798 and Unlimited-OCR's 0.7587. It specialised through two-stage training: supervised fine-tuning on Portuguese corpora, then DPO to stabilise inference. The author argues that concentrating parameters on one language remains a structural advantage despite emerging architectures.
Hume released Real World VoiceEQ, a speech evaluation benchmark covering over 40 proprietary and open-source voice models, more than 15 evaluation dimensions and over 60 metrics across ASR, TTS, S2S and speech understanding.
Import AI 464 reports Fable wrote what maintainers called the first genuine and fastest megakernel on KernelBench-Mega. CUDA on RTX PRO 6000 Blackwell achieved 18.71x acceleration, versus 14.4x for Claude Opus 4.8, 11.14x for GLM-5.2 and 4.34x for GPT 5.5.
Hugging Face released ScarfBench (Self-Contained Application Refactoring Benchmark) to evaluate AI agents migrating enterprise Java applications between Spring, Jakarta EE and Quarkus.
Every Eval Ever (EEE) and Hugging Face Community Evals are now interoperable, allowing cross-publication of evaluations with links to complete records. EEE stores around 229,000 results across more than 22,000 models and 2,200 benchmarks from 31 reporting formats.
The UK AISI alignment team and Timaeus co-founded nonprofit Sequent, arguing alignment research is unprepared for superintelligence. It plans 40–80 full-time staff within two to three years and initial fundraising of $100–150 million.
WeatherNext, the AI weather model developed by Google DeepMind and Google Research, helped the US National Hurricane Center predict five days ahead with 80% confidence that Hurricane Melissa would make landfall in Jamaica at Category 5 strength. Confidence approached 100% three days ahead.
Jack Clark estimates in Import AI 455 a roughly 60% chance of AI R&D autonomously training successor models without human involvement by the end of 2028, and around 30% in 2027.
Google released Vantage, a research experiment using generative AI simulated conversations to assess future skills such as problem-solving and collaboration in school and university students, with English registration open on Google Labs. An Executive LLM dynamically guides dialogue and an AI Evaluator scores against rubrics. A study with New York University involving 188 US participants aged 18–25 found AI–expert scoring agreement close to agreement between two human experts.
Google Research released ConvApparel, with over 4,000 human–machine multi-turn clothing-shopping conversations, nearly 15,000 turns, to quantify realism gaps between LLM user simulators and real people.
Google Research proposed situational judgement tests (SJT) adapted from psychological questionnaires such as IRI and ERQ to evaluate alignment between LLM behaviours and human consensus in real user–assistant scenarios.
Google Research studied trade-offs between evaluation item counts and annotators per item in Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation. Common counts of one, three or five annotators are often insufficient; reflecting human disagreement frequently requires more than ten.
Google and Cornell published a PNAS study asking GPT-4o, Perplexity, Claude 3.5, Gemini Advanced Pro 1.5, NotebookLM and a custom RAG system 67 expert superconductivity questions. Twelve international experts blindly assessed balance, comprehensiveness, concision, evidence, image relevance and qualitative feedback.
Mistral proposes judge LLMs scoring generator responses on numerical, binary or qualitative scales, then taking weighted averages across evaluation datasets to assess RAG systems.