GDPR Article 17 gives users a "right to erasure" — a service must be able to remove a person's data, and any model trained on it, on request. Naively that means retraining the whole model from scratch every time: O(N) work per deletion.
Logistic regression: p(x) = sigma(w1*x1 + w2*x2 + b)
Cross-entropy loss: L = -[y*ln(p) + (1-y)*ln(1-p)]
Gradient step: w <- w - eta * mean((p - y) * x)
SISA training (Sharded, Isolated, Sliced, Aggregated — Bourtoule et al. 2021) partitions the training set into S disjoint shards, each with its own model trained independently by real gradient descent. Predictions are the average of the shard models' probabilities. To forget one point, only the shard that contained it needs to retrain — everyone else is untouched:
naive retrain cost ~ N (every point, every deletion)
SISA retrain cost ~ N / S (one shard only)
expected savings ~ 1 - 1/S
- Click a sphere — files an erasure request for that training point (highlighted gold).
- Process erasure (SISA) — removes the point and retrains only its shard with real gradient descent; the colored floor (each shard's decision surface, averaged into the ensemble) updates live.
- Shards S — more shards means cheaper future deletions but coarser per-shard models, so test accuracy trades off against erasure cost. Changing it re-trains every shard once.
- Class separation — how cleanly the two classes are linearly separable; lower values inject more label noise and generate a new dataset.
- The vertical translucent walls mark shard boundaries in feature space; sphere color is the true class, floor color/height is the ensemble's predicted probability of class 1.