Batch size — the number of training examples used to compute each gradient update — is one of the most consequential hyperparameters in deep learning. This scene shows a ball (the model's parameters) descending a 3D loss landscape. Each step, the ball samples a mini-batch of points from the training data plane and moves downhill using the gradient estimated from that batch, plus random noise standing in for sampling variance.
1 / sqrt(batch size), matching the statistics of mini-batch SGD.Very large batch sizes can reach a similar final loss but often generalize slightly worse than smaller batches — a phenomenon researchers link to large-batch training converging to sharper minima, while the noise in small-batch SGD acts as an implicit regularizer that favors flatter, more robust minima.
A ball representing a model's parameters descends a 3D loss landscape, sampling mini-batches from a training data plane below — watch how batch size trades off gradient noise, convergence speed, and the ability to escape local minima.
Gradient noise scales as roughly 1/√(batch size). Small batches jitter but can hop out of shallow local minima; large batches converge smoothly but may settle into the nearest basin.
Adjust batch size and learning rate, then watch the highlighted mini-batch dots on the data plane and the ball's path on the loss surface update live. Reset to try a different starting point.
Research on "sharp vs. flat minima" suggests large-batch training tends to converge to sharper minima that generalize slightly worse, while small-batch noise acts as an implicit regularizer.