Training a machine learning model in a Docker container is easy — packaging code, weights and a CUDA runtime into one portable image. The hard part starts when that container needs to run on a shared Kubernetes cluster: the scheduler has to find a node with enough free GPUs, not just enough CPU and memory. This scene visualises a small cluster of nodes, each with a fixed GPU capacity, receiving a stream of simulated training jobs that request a number of GPUs.
Insufficient nvidia.com/gpu failure in Kubernetes.Kubernetes treats GPUs as an "extended resource" that cannot be shared or fractionally allocated by default — a pod either gets a whole GPU or none, which is exactly why bin-packing schedulers matter so much for keeping expensive accelerators from sitting idle in fragmented slivers across many nodes.
A 3D Kubernetes cluster receives a stream of simulated ML training containers, each requesting a number of GPUs, while a GPU-aware scheduler decides in real time whether to place, queue, or reject each job.
GPUs are indivisible "extended resources" in Kubernetes — a scheduler must find a node with enough entirely-free GPU slots before it can bind a pod, unlike CPU or memory which can be split fractionally.
Adjust job arrival rate, GPUs requested per job, GPUs per node, cluster size and the scheduling strategy, then watch jobs get placed on node racks, wait in the pending queue, or get rejected outright.
Bin-packing schedulers deliberately fill the fullest node that still fits a job, which fragments GPUs less than round-robin "spread" placement — critical when autoscalers try to drain and shut down idle nodes.