The engineering of running models: inference optimisation, memory and cost, serving architecture and infrastructure choices.
Claude Sonnet 5.5 is now available on Amazon Bedrock and Claude Platform on AWS. It's a smarter, more efficient Sonnet model for focused coding and knowledge work, with a lower cost per task at faster speed. This post covers what's new, when to choose Sonnet, and how to get started.
Robots are getting smarter, but how can their hardware match that growth? New Microsoft Research findings show that moving AI inference beyond the robot can improve task success, boost efficiency, and support more advanced physical AI workloads.
To build and deploy sophisticated robotics applications that can perceive, reason and act in dynamic environments, developers need new physical AI models and tools. The ROS open framework is a project from Open Robotics that helps humans build robots.
The definitive Jev podcast with its lead creator.
A rewrite this size wasn't affordable before agents. Here's what porting the Copilot agent runtime to 800,000 lines of production Rust actually took.
System performance, efficient infrastructure scaling and continuous software optimization are key levers that determine AI inference economics. Higher system performance means more tokens generated, resulting in higher revenue. Efficient scaling means throughput grows proportionally as hardware gets added, requiring fewer resources to serve users at scale. Continuous optimization means generating more value from infrastructure investments.
On a sweltering August evening in Silicon Valley, as the sun dropped and air conditioning loads spiked, Silicon Valley Power sent a signal to an AI factory to adjust its power consumption.
Ian Buck, vice president of hyperscale and high-performance computing at NVIDIA, Tuesday spoke on AI factory efficiency at the AI Infra Summit, the Santa Clara Convention Center event that has morphed into a Coachella of infrastructure tech.
In controlled offline evaluations, HydraFusion’s selective coding workflows matched or exceeded the evaluated Opus 5 baseline while reducing estimated workflow cost. Now available as a research preview in GitHub Copilot.
Hugging Face’s WebAI team released @huggingface/kernels, a lightweight library for loading and running optimised WebGPU kernels from the Hugging Face Hub. An initial collection of 207 kernels is also available as separate repositories under Apache-2.0.
Liquid AI released DSpark draft model checkpoints for LFM2.5-1.2B-Instruct, LFM2.5-2.6B and LFM2.5-8B-A1B. Speculative decoding accelerates decoding without changing output quality, increasing throughput by up to 3.18 times on GPUs and 2.87 times on devices.
A Hugging Face blog tutorial demonstrates a streaming data loop for Strands Robots. The same Robot() object records demonstrations, syncs them to a Storage Bucket, streams training data from the Hub and deploys the checkpoint back to hardware, keeping the LeRobot disk format unchanged throughout.
NVIDIA released Magpie TTS Multilingual, a 364M-parameter open-weight speech synthesis model supporting English, Spanish, French, German, Italian, Vietnamese, Chinese, Hindi and Japanese, plus newly added Modern Standard Arabic, Korean and Brazilian Portuguese, for 12 languages in total.
Hugging Face announced that vLLM’s transformers modelling backend now matches or exceeds the throughput of vLLM’s handwritten native implementations across several LLM architectures. It uses torch.fx for static graph analysis and ast to rewrite source code, dynamically applying inference-related layer fusion at runtime to match custom-code performance.
At Build 2026, Microsoft announced Foundry Managed Compute and a Hugging Face model collection, with weekly updates to open-weight models and one-click deployment to Foundry's managed GPU platform.
Hugging Face and SkyPilot released an integration allowing a Hugging Face Bucket or any model, dataset or Space repository to be mounted into SkyPilot tasks with an hf:// URL and an existing HF_TOKEN, running across more than 20 clouds, Kubernetes, Slurm and local environments.
Google Research published an architecture attaching multi-token prediction (MTP) heads to frozen Gemini Nano v3 models, accelerating on-device inference without changing backbone weights. It has shipped with the Pixel 9 and 10 series.
Mistral AI added new Connectors capabilities. Admin controls for workspace- or organisation-level access and individual tool toggles are generally available, as are API keys scoped to connectors.
Mistral OCR 4 delivers enterprise document AI with 170-language support, bounding boxes, and self-hosted deployment.
Google DeepMind released experimental open-source model DiffusionGemma, generating text blocks in parallel through text diffusion and achieving up to 4 times faster inference on dedicated GPUs.