Few-Shot Learning: Classifying From Almost No Labelled Data

The problem: barely any labels

Few-shot learning tackles the setting where each class has only a handful of labelled examples — the k-shot "support set" — far too few to train a conventional model from scratch. Prototypical networks, one of the simplest and most effective approaches, sidestep the data-hunger problem with a strikingly simple idea.

Prototypes: one point per class

Compute each class's prototype — the mean of its support examples in embedding space — then classify any new query point by which prototype it's closest to. No further training is needed once the prototypes are computed; classification is just nearest-neighbour lookup against a handful of averaged points.

The assumption doing all the work

This only works because the embedding space itself — the coordinates points sit at — isn't raw pixels or raw features, but the output of a network already pretrained, often on a large related task, to place semantically similar things near each other. Given a good embedding space, even 1-5 labelled examples per class are often enough to locate a useful prototype, because the hard work of learning what "similar" means was already done upstream, before few-shot classification ever begins.

Diminishing returns on more shots

Increasing the number of shots per class averages more support points into each prototype, reducing noise in its estimated position. But returns diminish quickly: going from 1 shot to 3 typically helps substantially, while going from 10 shots to 15 barely moves accuracy once the prototype has already converged close to the true class center.

Where this shows up in practice

Rapidly adding new product categories to an image classifier, personalizing a voice assistant to a new speaker from a handful of samples, and any setting where collecting hundreds of labelled examples per new class simply isn't feasible are the natural home for few-shot methods built on this prototype idea.