Hugging Face shows streaming MoE expert inference with Transformers
What happened
AI digest
On 18 November 2026, the Hugging Face blog published a post on implementing streaming loading of MoE experts with Transformers. On a MacBook M5 with 32GB of unified memory, the author ran the Qwen3.8-Flash-Next model, whose FP8 experts total 121GB. In testing, short decoding speed was 13.9 tok/s, about 6% slower than Strata's 14.8 tok/s; prefill speed was 228 tok/s, slightly faster than Strata's 220 tok/s. The report does not mention whether the approach had earlier versions or prior progress.
Written by AI from the reports · updated 2 h ago
Timeline
Follow the reports to see the event from each side.
Nov 18
- Hugging Face Blog180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers
Streaming MoE experts directly from SSD gets within 6% of Strata on token generation.
Heat of this event
Not enough continuous observations yet to draw a trend.