Domain-Adversarial Transfer Learning: The Gradient Reversal Layer

Adaptation Controls

Gradient-reversal strength λ 0.80

Learning rate η 0.25

Initial domain shift ‖D‖ 2.20

Live Metrics

Domain-classifier acc.

—

Target task acc.

—

Training steps

Legend

Source · class A Source · class B Target · class A (true label, unseen) Target · class B (true label, unseen) Domain-classifier boundary plane

How it works

A feature extractor maps source data (labelled, spheres) and target data (unlabelled, octahedra) into the same embedding space. A domain shift D initially offsets the whole target cloud, so a classifier trained only on source features (decision plane x = 0) misreads target points whose x has crossed the boundary — this is the target-task-accuracy readout.

domain classifier:  p = σ(w·x + b)
domain loss:        L_d = −[y·log p + (1−y)·log(1−p)]
∇ₓ L_d = (p − y)·w

Gradient Reversal Layer (Ganin & Lempitsky, 2015):
during backprop the feature extractor receives −λ·∇L_d
instead of +λ·∇L_d, i.e. it ASCENDS the domain loss
while the domain classifier keeps DESCENDING it normally.

λ — reversed-gradient strength. λ = 0 is plain source-only training (no adaptation); very large λ can overpower the task signal and blur class separation even on the source side — a real DANN failure mode, try λ ≈ 2.
η — the shared gradient step size, applied every simulated step to both the domain classifier's weights and the feature positions.
Domain shift ‖D‖ — how far the target cloud starts from source (only takes effect on reseed); larger shift = harder adaptation problem.
Source points additionally feel a fixed task-consistency spring toward their two class prototypes (labels are known for source) — target points get no such spring, only the adversarial pull, exactly as in real unsupervised domain adaptation where target labels don't exist during training.

Real-world relevance: this is the core trick behind adapting a classifier trained on synthetic or studio images to real-world photos, or a model trained on one hospital's scanners to another's, without ever labelling the new domain.

Source domain (labelled) Target domain (unlabelled)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.