Both classifiers learn a linear score f(x) = w·x + b over the same 2D points, but they are trained with different loss functions of the margin m = y·f(x), y ∈ {−1,+1}:
Logistic / cross-entropy: L_ce(m) = ln(1 + e^(−m))
Hinge (linear SVM): L_hinge(m) = max(0, 1 − m)
Cross-entropy is smooth and never reaches exactly zero — it keeps pushing every point to be more confident, even ones already classified correctly. Hinge loss is zero for any point already past the margin (m ≥ 1), so the SVM boundary is shaped only by the points closest to it (the support vectors) while confident points stop contributing gradient at all. The small bars drawn under each point are the real |∂L/∂f| magnitude for that point under each loss — watch the orange hinge bars vanish completely for points safely on the correct side, while the cyan cross-entropy bars only ever shrink, never truly hit zero.
Full-batch gradient descent, N points:
w ← w − η · ( (1/N)Σ ∂L/∂w + λw )
b ← b − η · (1/N)Σ ∂L/∂b
∂L_ce/∂f = σ(f) − y⁺ (y⁺ = 1 if y=+1 else 0, σ = sigmoid)
∂L_hinge/∂f = −y if m < 1, else 0 (subgradient)
- Play / Step — run full-batch gradient descent on both models simultaneously with the same learning rate η.
- Class overlap — regenerates the two Gaussian clusters with more or less spread, controlling how separable the data is.
- L2 regularization λ — shrinks both weight vectors each step, keeping the SVM margin from collapsing to zero on noisy data.
- The loss-curve strip plots L_ce(m) and L_hinge(m) directly — the two bright dots riding along them are each model's current average margin, showing exactly where on its own loss curve each classifier currently sits.
- Hinge-zero points counts how many points currently contribute exactly zero gradient to the SVM (m ≥ 1) — that count is always 0 for cross-entropy, since its gradient is never exactly zero.
Real-world relevance: this exact choice — cross-entropy for logistic regression and neural network classifiers, hinge loss for SVMs — is one of the first modeling decisions in any classification pipeline, and it directly explains why SVMs are "sparse" (only support vectors matter) while neural nets keep adjusting on every example.