AWS Machine Learning Blog· Joseph Cottingham·· 7 h ago
Stop guessing auto scaling targets for TTS on Amazon SageMaker AI
Stop guessing auto scaling targets for TTS on Amazon SageMaker AI
AI summary
Teams deploying custom text-to-speech models on Amazon SageMaker AI often guess at autoscaling thresholds, and a wrong guess either wastes GPU spend or breaches a latency SLO. This post deploys Kokoro-82M as a custom container and derives a measured autoscaling policy from two live-endpoint measurements, Q_max and T_total, instead of guesswork.
Selection record
Threshold 60Official, first-handFirst 31Second 31
Not admittedSum of both 62 < twice the threshold 120
- Source tier
- Official, first-hand; this tier's threshold is 60
- Pre-filter
- passed：AWS博客讲TTS模型部署与自动扩缩容实测
A model scores each item twice, independently, against one written standard, out of 100. An item is admitted only when the two scores add up to twice the threshold. The threshold is set per source tier.
Source: AWS Machine Learning Blog · aws.amazon.com