A trained model needs to run somewhere that clients can reach reliably, at whatever scale demand requires. This scene shows the pieces of a typical managed inference deployment: incoming requests arrive at a load balancer, which spreads them across a fleet of identical serving pods, each holding a copy of the packaged model.
Canary releases are named after the "canary in a coal mine" — miners once carried canaries underground because the birds showed signs of gas poisoning before humans did, giving an early warning. A canary deployment plays the same role for a bad model rollout, catching problems while only a small slice of traffic is exposed.
A load balancer routes a stream of inference requests across a ring of model-serving pods. Watch the fleet autoscale with traffic and roll out a new model version using rolling, canary, or blue-green deployment.
How request traffic maps to pod count under autoscaling, how estimated tail latency rises under load, and how three real deployment strategies mix old and new model versions differently while a rollout is in progress.
Raise traffic to see the pod ring scale, pick a deployment strategy, then click Deploy v2 (or drag the canary weight slider) to send a new model version live and watch requests shift from blue pods to amber ones.
Canary releases are named after canaries once carried into coal mines to give miners early warning of danger — a canary deployment gives engineers the same early warning before a bad model reaches every user.