A pretrained network is a stack of layers. Early layers learn general features (edges, colors, textures) that transfer well to almost any task; late layers learn features specific to the source task. Transfer learning freezes the general layers 1…k, replaces layers k+1…10 with fresh random weights, and trains only the new part on the target dataset.
specificity(i) = (i / 10)^1.5 // grows with depth
scratchAcc(D) = 45 + 40 · frac(D) // baseline, frac = log-scaled dataset size in [0,1]
transferBonus(k) = 18 · (1 − frac) · mean(1 − specificity(1..k))
coAdaptDip(k) = (fineTune ? 2 : 14) · sin(π·k/10)
fineTuneBonus = fineTune ? 6 · frac : 0
accuracy = clamp(scratchAcc + transferBonus − coAdaptDip + fineTuneBonus, 0, 99)
- Split layer — how deep the frozen, transferred backbone goes. Freezing more of the general early layers helps most when the target dataset is small; freezing too far into the network drags in source-specific features that don't transfer as well.
- Co-adaptation dip — splitting mid-network without fine-tuning breaks the fragile interdependence between adjacent layers (Yosinski et al., 2014), producing a real accuracy dip that peaks around the middle split — the dip term is exactly zero at k=0 and k=10 (both ends of the stack) and largest near k=5, which is why the sin(πk/10) shape was kept unchanged from the source model: it already matches the described phenomenon (verified numerically in a scratch check across k=0..10, frac=0..1, both fine-tune states — no negative accuracies, dip vanishes only at the two endpoints).
- Dataset size — with very little target data, transferring features beats training from scratch by a wide margin; with a lot of data, the gap narrows since a from-scratch network can learn its own good features.
- Random-init baseline — the accuracy curve you'd get training all 10 layers from scratch on the target data alone, for comparison.
This is a simplified, formula-driven model of a real and widely measured phenomenon in deep learning, not a live neural-network training run.