Cross-Entropy vs. Hinge Loss: How Loss Choice Shapes a Classifier (2D)

Two linear classifiers — a logistic-regression model trained with cross-entropy loss and a linear SVM trained with hinge loss — learn the same 2D dataset side by side on a flat canvas. Full-batch gradient descent runs on both models from the same random initialization and the same data every step, with their decision boundaries redrawn live, a shared loss-curve strip plotting both loss functions against the margin with a moving dot per model, and a real per-point gradient-magnitude bar under every point showing exactly how many points hinge loss has already stopped updating on (zero gradient, m ≥ 1) versus cross-entropy, which never fully stops. Adjust the learning rate, class overlap, and L2 regularization to see exactly why cross-entropy keeps refining confident predictions while hinge loss stops caring the moment a point clears the margin.