How Convolution Lets a Neural Network See

An image is just a grid of numbers

To a computer, a photograph is nothing more than a grid of numbers. A greyscale image is a two-dimensional array where each cell (each pixel) holds a single intensity value, typically from 0 (black) to 255 (white). A colour image adds a third dimension: three stacked grids, one each for red, green and blue intensity, so a modest 224 by 224 pixel colour photo is really an array of 224 × 224 × 3, or roughly 150,000 individual numbers. Any computer vision system, however sophisticated, ultimately has to extract meaning from nothing but this grid of numbers, with no built-in notion of edges, shapes or objects.

Before deep learning, computer vision relied on hand-designed operations — filters engineered by researchers to detect specific patterns like edges or corners, such as the Sobel or Canny operators. These worked, up to a point, but required a human to guess in advance which patterns mattered. Convolutional neural networks (CNNs) replace that guesswork: instead of a person designing the filters, the network learns them directly from labelled examples, discovering whatever visual patterns turn out to be useful for the task at hand.

The convolution operation, step by step

A convolutional layer works by sliding a small grid of learnable numbers, called a filter or kernel — commonly 3×3 or 5×5 — across the image, one position at a time. At each position, the filter's values are multiplied element-by-element with the pixel values directly beneath it, and all the products are summed into a single number. That single number becomes one pixel of the output, called a feature map. Slide the filter over every position in the image and you get an entire feature map: a new, usually smaller grid where each value summarises how strongly that filter's pattern was present at that location in the original image.

Concretely, if a 3×3 filter has been trained to detect vertical edges, it will contain large positive numbers down one column and large negative numbers down another; wherever the image has a genuine vertical edge (a sharp brightness transition running top to bottom), the element-wise multiplication and sum produces a large output value, and wherever the image is flat or has a horizontal edge instead, the output stays close to zero. A single convolutional layer typically learns dozens of different filters at once — some tuned to horizontal edges, some to diagonal edges, some to colour transitions or small textures — and stacks all of their feature maps together as the layer's output.

Three settings control exactly how the sliding happens. Stride is how many pixels the filter jumps between positions; a stride of 1 checks every possible position and produces a large, detailed feature map, while a stride of 2 skips every other position, halving the output size and roughly quartering the computation. Padding decides what happens at the image's edges: with "valid" padding the filter only slides where it fits entirely inside the image, which shrinks the output slightly with every layer, while "same" padding adds a border of zeros around the image so the output stays the same size as the input. And the output size follows directly from these choices: for an input of size H, a filter of size K, stride S and padding P, the output dimension is (H + 2P − K) / S + 1.

Why sliding one small filter everywhere is so powerful

The convolution operation has two properties that make it dramatically more efficient than a fully-connected layer for image data, and both come directly from the fact that the same small filter is reused at every position. The first is parameter sharing: a 3×3 filter has only 9 weights (plus a bias term) no matter how large the input image is, because that same set of 9 numbers gets reused at every sliding position rather than each position getting its own independent set of weights. A fully-connected layer processing a 224×224 image would need tens of millions of weights just for one layer; a convolutional layer with a handful of 3×3 filters needs only a few hundred.

The second property is translation invariance. Because the filter is applied identically everywhere in the image, a pattern the filter has learned to detect — say, a curved edge that is part of a bee's wing — will be detected wherever it appears in the frame, whether that is the top-left corner or dead centre. A fully-connected network would effectively have to relearn what a wing edge looks like separately for every possible position, which is both wasteful and a recipe for poor generalisation to images where the object has shifted slightly.

Pooling layers, usually inserted between convolutional layers, complement this by deliberately shrinking the feature maps — most commonly with max pooling, which slides a small window (typically 2×2) over the feature map and keeps only the largest value in each window, discarding the rest. This reduces the amount of computation needed downstream, makes the network more robust to small shifts or distortions in exactly where a feature appears, and progressively enlarges the effective receptive field: the area of the original image that ultimately influences each value deep in the network.

From edges to eyes to faces: the hierarchy of features

A single convolutional layer can only detect very local, simple patterns, because its filter is small and only sees a small patch of the image at a time. The real power of a CNN comes from stacking many convolutional layers on top of each other. The first layer's output — feature maps showing where edges and simple colour transitions occur — becomes the input to a second convolutional layer. Because that second layer's filters operate on the first layer's feature maps rather than the raw pixels, and because pooling has enlarged the effective receptive field, the second layer can combine several nearby edges into a more complex pattern, such as a corner, a curve, or a small texture patch.

This compounds with depth. By the third or fourth layer, filters commonly respond to recognisable parts — something like "round shape with a dark centre" that resembles an eye, or a wavy stripe pattern. By the deepest layers of a large network, filters can respond to entire object parts or whole categories: a wheel, a wing, a face. This is why CNN architectures are described as learning hierarchical features automatically: nobody tells the network what an eye looks like; it is discovered purely because "detecting eye-like patterns" turns out to be a useful intermediate step on the way to correctly classifying photos of animals or faces, given enough labelled training examples and enough layers to build the hierarchy up gradually.

This layer-by-layer buildup is exactly the kind of process that benefits from an interactive, step-through visualization: watching a single 3×3 filter slide across a small image grid and produce one output value at a time makes the mechanical operation concrete, and watching several such feature maps stack and feed into the next layer makes the hierarchy-of-features idea tangible in a way that a static diagram cannot.

From filters to famous architectures

Modern CNN architectures are essentially different strategies for arranging convolution and pooling layers efficiently. AlexNet, which won the 2012 ImageNet competition and helped trigger the deep learning boom in computer vision, stacked five convolutional layers with large early filters. VGG showed that stacking many small 3×3 filters (rather than fewer large ones) could reach greater effective receptive fields with fewer parameters and better performance, at the cost of depth. ResNet solved a problem that emerged once networks got very deep — gradients vanishing as they propagate back through dozens of layers — by adding skip connections that let a layer's output be added directly to its input, allowing networks with over a hundred layers to train successfully. More recent architectures, including EfficientNet and vision transformers (which replace convolution with a self-attention mechanism borrowed from language models), continue to refine the trade-off between accuracy, model size and inference speed, but the sliding-filter convolution operation described here remains the foundational building block that made deep computer vision practical in the first place.