Both classifiers learn a linear score f(x) = w·x + b over the same 2D points, but they are trained with different loss functions of the margin m = y·f(x), y ∈ {−1,+1}:
Logistic / cross-entropy: L_ce(m) = ln(1 + e^(−m))
Hinge (linear SVM): L_hinge(m) = max(0, 1 − m)
Cross-entropy is smooth and never reaches exactly zero — it keeps pushing every point to be more confident, even ones already classified correctly. Hinge loss is zero for any point already past the margin (m ≥ 1), so the SVM boundary is shaped only by the points closest to it (the support vectors) while confident points stop contributing gradient at all.
Full-batch gradient descent, N points:
w ← w − η · ( (1/N)Σ ∂L/∂w + λw )
b ← b − η · (1/N)Σ ∂L/∂b
∂L_ce/∂f = σ(f) − y⁺ (y⁺ = 1 if y=+1 else 0, σ = sigmoid)
∂L_hinge/∂f = −y if m < 1, else 0 (subgradient)
- Play / Step — run full-batch gradient descent on both models simultaneously with the same learning rate η.
- Class overlap — regenerates the two Gaussian clusters with more or less spread, controlling how separable the data is.
- L2 regularization λ — shrinks both weight vectors each step, keeping the SVM margin from collapsing to zero on noisy data.
- The floating curves behind the data plane plot L_ce(m) and L_hinge(m) directly — the two bright dots riding along them are each model's current average margin, showing exactly where on its own loss curve each classifier currently sits.
Real-world relevance: this exact choice — cross-entropy for logistic regression and neural network classifiers, hinge loss for SVMs — is one of the first modeling decisions in any classification pipeline, and it directly explains why SVMs are "sparse" (only support vectors matter) while neural nets keep adjusting on every example.