🧮 Pipeline Parallelism Bubble Lab: GPU Stages, Micro-batches and Idle Time
Pipeline-parallel training of one model across p GPU stages with m micro-batches: GPipe and 1F1B schedules simulated as a dependency graph, bubble fraction (p-1)/(m+p-1), makespan (m+p-1)(tf+tb), peak stored activations m versus min(p,m), with Blender-built GPU cards.
⚙ Under the hood
Pipeline-parallel training of one model across p GPU stages with m micro-batches: GPipe and 1F1B schedules simulated as a dependency graph, bubble fraction (p-1)/(m+p-1), makespan (m+p-1)(tf+tb), peak stored activations m versus min(p,m), with Blender-built GPU cards.
pipeline parallelismGPipe1F1Bpipeline bubbledistributed trainingmicro-batchGPUBlender
3D · Three.js / WebGL renderer · 60 FPS target · runs fully client-side, no install