SageMaker AI shares MoE kernel engineering optimisation
What happened
On 13 October 2026, the AWS Machine Learning Blog published an article on MoE kernel engineering optimisation practices on Amazon SageMaker AI. The article notes that during the decode phase, MoE models suffer from a small M dimension, which leads to low Hopper GPU utilisation. To address this, AWS proposes the Swap-AB optimisation, which promotes the expert output dimension N=8192 to the first operand, delivering a 2–4% end-to-end throughput improvement for large-scale MoE models on H200. The optimisation has been merged into vLLM 0.24.0 and later versions.
Written by AI from the reports · updated 3 d ago
Timeline
Follow the reports to see the event from each side.
- AWS Machine Learning BlogMixture-of-Experts kernel engineering on Amazon SageMaker AI
AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.
Heat of this event
Current heat 3·Peak in the comparable range 10（Oct 14 00:00）·Change over 24 hours in the comparable range -50%
The trend compares only the same accounts observed without a break, so its range can be narrower than the current heat count. Move the pointer or tap the chart to see each hour; with a keyboard, use the left and right arrow keys.