Progress in AI speech and audio: speech synthesis, real-time conversation, music generation and audio understanding.
Two weeks ago, we opened up Phonon, our small text-to-speech model that runs on your own device. It needs no server and no network, and what you type stays with you.
One open model for multilingual transcription, speaker attribution, translation, and audio Q&A, with top leaderboard results, plus the open Granary v2 data pipeline and fine-tuning recipes to adapt it.
Voxtral 3: Mistral's voice agent SDK for real-time speech in 50+ languages, accurate transcription, emotional TTS, deep tool use.
Introduced last spring, the Granite Switch project enables a modular approach to model development. Now Granite Switch 4.1 models, armed with a Decision adapter and the fastest ASR model on the OpenASR leaderboard, can move efficiently between natural-language responses, general and specialized decisions, and speech-to-text tasks—all from one hosted endpoint.
Google DeepMind released Gemini 3.8 Live with Live Avatar, adding near-real-time video generation to a native live conversation model to create a dynamic visual avatar that can hear, see and speak. It is available on Gemini Enterprise from today.
Meta positioned Muse as a personal agent at Connect 2026 and announced hardware and feature updates around it. Muse supports voice and live video, runs tasks in the background and will come to all Meta glasses. Each Muse has its own email address, the Mac version supports computer use, and its connector platform has more than 1,500 apps.
Google DeepMind released Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS. The former targets character design and line-by-line performance direction, while the latter targets high-concurrency voiceovers and voice agents.
Google DeepMind released two real-time conversation models, Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking. The former targets scale and cost efficiency, while the latter targets complex tasks and multistep reasoning.
GPT‑Live‑1 brings natural, full-duplex voice conversations to the API, with stronger instruction following, custom voices, and telephony support.
Voice Arena and Hugging Face added Monsoon en-IN and Monsoon hi-IN evaluation datasets to Open ASR Leaderboard, making Hindi its first Indian language.
Now you can get more intelligent speech-to-text transcription with Gemini 3.5 Transcribe.
Hugging Face proposed three tests to quantify benchmark optimisation in speech recognition: a consensus-disagreement probe, masked-entity retrieval and spelling switches. Evaluating 11 open-source ASR models, it found some high-scoring models reproduce errors in VoxPopuli and LibriSpeech reference transcripts even when contradicted by audio, when relevant words are muted or when both spellings fit the audio.
Google Research released AMIE (Video), built on Gemini and Project Astra, for real-time video clinical consultations. It can perceive non-verbal cues and guide virtual physical examinations.
NVIDIA released Magpie TTS Multilingual, a 364M-parameter open-weight speech synthesis model supporting English, Spanish, French, German, Italian, Vietnamese, Chinese, Hindi and Japanese, plus newly added Modern Standard Arabic, Korean and Brazilian Portuguese, for 12 languages in total.
Google DeepMind released its next-generation music model Lyria 3.5 on Google Flow Music. It claims improvements in musicality, lyrics, vocals and creative control, including more complex and natural melodic structures, lyrics with better prompt adherence and structural awareness, more expressive vocals with clearer pronunciation, and easier control of rhythm and duration.
Hume released Real World VoiceEQ, a speech evaluation benchmark covering over 40 proprietary and open-source voice models, more than 15 evaluation dimensions and over 60 metrics across ASR, TTS, S2S and speech understanding.
Hugging Face and Cerebras jointly demonstrated a real-time speech-to-speech pipeline, accelerating Gemma 4 31B inference with Cerebras and combining Nvidia Parakeet speech recognition with Alibaba Qwen3TTS synthesis.
Gemini 3.5 Live Translate brings near real-time, natural speech translation to Google AI Studio, Google Translate and Google Meet.
Researching the path to AI-augmented care and development of an AI co-clinician.
Our newest audio model introduces granular audio tags that give you precise control to direct AI speech for expressive audio generation.