Domain-Adversarial Transfer Learning: The Gradient Reversal Layer (2D)

Adaptation Controls

Gradient-reversal strength λ 0.80

Learning rate η 0.25

Initial domain shift ‖D‖ 2.20

Domain classifier lr 0.90

Task-consistency spring k 1.10

Feature jitter 0.03

Live Metrics

Domain-classifier acc.

—

Target task acc.

—

Training steps

Legend

Source · class A (filled circle) Source · class B (filled circle) Target · class A (true label, unseen — diamond) Target · class B (true label, unseen — diamond) Domain-classifier boundary line

How it works

A feature extractor maps source data (labelled, circles) and target data (unlabelled, diamonds) into the same 2D embedding space. A domain shift D initially offsets the whole target cloud, so a classifier trained only on source features (decision line x = 0) misreads target points whose x has crossed the boundary — this is the target-task-accuracy readout.

domain classifier:  p = σ(w·x + b)
domain loss:        L_d = −[y·log p + (1−y)·log(1−p)]
∇ₓ L_d = (p − y)·w

Gradient Reversal Layer (Ganin & Lempitsky, 2015):
during backprop the feature extractor receives −λ·∇L_d
instead of +λ·∇L_d, i.e. it ASCENDS the domain loss
while the domain classifier keeps DESCENDING it normally.

λ — reversed-gradient strength. λ = 0 is plain source-only training (no adaptation); very large λ can overpower the task signal and blur class separation even on the source side — a real DANN failure mode, try λ ≈ 2.
η — the shared gradient step size, applied every simulated step to both the domain classifier's weights and the feature positions.
Domain shift ‖D‖ — how far the target cloud starts from source (only takes effect on reseed); larger shift = harder adaptation problem.
Domain classifier lr — how fast the adversary itself learns to separate the two clouds; a slow classifier gives the reversed gradient a weaker, staler signal to fight.
Task-consistency spring k — how hard labelled source points are pulled back toward their own class prototype every step; target points get no such spring — only the adversarial pull — exactly as in real unsupervised domain adaptation where target labels don't exist during training.
Feature jitter — small random walk added to every point per step, standing in for stochastic mini-batch noise; push it up to see how robust the adaptation is to noisy gradients.
Drag the embedding-space view to pan, scroll/pinch to zoom — the accuracy-history strip below always keeps a live rolling window of both readouts vs. training step.

Real-world relevance: this is the core trick behind adapting a classifier trained on synthetic or studio images to real-world photos, or a model trained on one hospital's scanners to another's, without ever labelling the new domain.

Source domain (labelled) Target domain (unlabelled)

drag to pan · scroll to zoom