Skip to content
Hugging Face Blog·· 22 h ago

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

180B Parameters, 32GB of RAM: Pushing MoE Inference to the Limit with Transformers

AI summary

Streaming MoE experts directly from SSD gets within 6% of Strata on token generation.

Selection record

Not admittedSum of both 116 < twice the threshold 120

Source tier
Official, first-hand; this tier's threshold is 60
Pre-filter
passed:MoE推理优化与专家流式加载技术实践

A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.

Source: Hugging Face Blog · huggingface.co