Each glowing tower is a container built from a Docker image: stacked, read-only layers — base OS, CUDA/runtime, Python dependencies, model weights, and application code — sitting on top of each other. Every replica in the cluster runs from the same image, so identical layers only need to exist once on disk and are shared across containers, which is exactly how Docker's layer cache works in production.
--gpus all / NVIDIA Container Toolkit access to host accelerators from inside an isolated container.Because container image layers are content-addressed, pulling ten replicas of a model server usually transfers only a few hundred MB of unique application code — the multi-gigabyte base image and CUDA runtime are downloaded once and reused, which is why well-layered ML images scale out so much faster than monolithic VM images.
Watch a layered ML container image get built once and replicated across a GPU-backed cluster, with each layer's cache reuse and each replica's GPU passthrough rendered live in 3D.
Every replica is a stack of read-only layers — base OS, CUDA runtime, Python dependencies, model weights, app code. Identical layers are shared across containers via the image cache, so only the unique top layer differs per replica.
Scale replicas, change model weight size, and toggle layer caching and GPU passthrough. Watch the pull-size stat and beam brightness respond, and compare cached vs. uncached total transfer.
Because image layers are content-addressed, scaling out an inference service to ten pods usually re-downloads only the small application layer — the multi-gigabyte CUDA runtime and model weights are pulled once and reused.