Feature Pyramid Network: Multi-Scale Object Detection

Object Placed In Image

Object size 96 px

Position in image (u = v) 50%

Network Pathways

Scale Assignment

Assigned level

Stride

16 px

Feature-map resolution

14 × 14

How it works

A Feature Pyramid Network (Lin et al., 2017) builds a backbone of feature maps C2…C6 that shrink 2× at every stage, then a top-down pathway upsamples the coarsest map and merges it (element-wise add) with a 1×1-conv "lateral" projection of each backbone level, producing pyramid outputs P2…P6 that are all semantically strong AND multi-resolution.

Backbone strides:  4   8   16   32   64   (C2..C6)
Pyramid levels:    P2  P3  P4   P5   P6
Top-down:  P_i = Upsample2x(P_{i+1}) + Conv1x1(C_i)   (lateral connection)

Detectors that use an FPN neck (Faster R-CNN-FPN, RetinaNet, and the multi-scale heads in YOLO) assign each ground-truth box to exactly one pyramid level by its pixel area, using the rule from the original paper (k0 = 4, canonical scale 224):

k = round( k0 + log2( sqrt(w·h) / 224 ) ),   k clamped to [2, 6]

Object size slider — sets the object's side length in pixels; the formula above recomputes which level P2…P6 it lands on live.
Position slider — moves the object across the image; the highlighted cell tracks it on both the matching backbone map and the matching pyramid map.
Lateral connections OFF — the pyramid stack dims, standing in for what happens without the 1×1 lateral projection: fine spatial detail from the early backbone layers never reaches the pyramid, so small objects lose localization accuracy.
Top-down pathway OFF — the upsample-and-merge arrows disappear, representing raw backbone features that never receive the strong semantics of the coarser layers; large-context awareness at fine resolutions is lost.

Why it matters: a single feature map is a bad fit for object detection because a 32×-downsampled map is great at finding large objects but has almost no spatial resolution left for small ones. FPN's trick — reuse a cheap top-down pathway so every level is both high-resolution *and* semantically strong — is now standard in nearly all two-stage and one-stage detectors.

Backbone C2–C6 Pyramid P2–P6 Object / assigned cell

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.