〰️ Convolution — How Filters Reshape Signals

The impulse response h(t) is the output a system produces when its input is a single unit impulse (a Dirac delta, or a single spike in discrete systems). Because any arbitrary signal can be decomposed as a sum of scaled, time-shifted impulses, the output for any input is simply the convolution of that input with the impulse response. This means knowing h fully characterises any LTI system — microphones, amplifiers, rooms, and digital filters alike.

Convolution is ubiquitous: photo-editing software applies it for Gaussian blur, sharpening, and edge detection; audio workstations use it for reverb (convolving dry audio with a room's impulse response); wireless communications engineers model channel distortion with it; radiologists use it in CT and MRI reconstruction; and deep-learning frameworks build convolutional neural network layers directly on discrete 2-D convolution. Even the smoothing filter in your smartwatch's accelerometer is a convolution.

Yes — this is the Convolution Theorem. If F and G are the Fourier transforms of f and g, then the Fourier transform of f*g is simply F times G (pointwise multiplication). This is why the Fast Fourier Transform (FFT) makes large convolutions dramatically faster: instead of O(N^2) multiply-add operations in the time domain, FFT-based convolution takes only O(N log N) steps. The Gaussian blur in your photo app and the FIR filters in audio hardware exploit this fact.

Not necessarily. A wider kernel averages or integrates over more of the signal, which reduces noise and smooths edges — but it also smears fine detail and delays causal filters (like the exponential echo kernel) by more samples. Choosing kernel width is a trade-off between noise suppression and resolution. In image processing, over-blurring destroys texture; in audio, excessive reverb tails reduce intelligibility. The right width depends entirely on the application.

The integral form of convolution appeared in 18th-century analysis, with contributions from Euler, Laplace, and Cauchy in the context of differential equations and integral transforms. The Fourier transform connection was established by Joseph Fourier in the early 19th century. The engineering application to linear filters was systematised by Oliver Heaviside in the late 1800s and later by Claude Shannon and Norbert Wiener in the mid-20th century as part of information theory and cybernetics.

Cross-correlation (f star g)(t) = integral f(tau) * g(t + tau) d-tau is almost identical to convolution but without the kernel flip. When f = g (autocorrelation) it measures self-similarity at different lags. In template matching and pattern recognition, cross-correlation locates where a template best fits a signal. Convolution and cross-correlation differ only in the sign of the shift; for symmetric kernels (like a Gaussian) they produce identical results. Related simulations include Fourier Transform, Low-Pass / High-Pass Filtering, and Impulse Response.

In a CNN, each convolutional layer applies a small 2-D kernel (e.g. 3x3 or 5x5 pixels) to an input feature map using the discrete convolution sum. The kernel weights are not hand-designed — they are learned from data via backpropagation. Stacking many such layers lets the network learn hierarchical features: edges in layer 1, textures in layer 2, object parts in layer 3, and so on. The same mathematical operation that blurs a photo or echoes a sound is therefore also the engine behind image classifiers, object detectors, and speech recognition systems.

Active research directions include: depthwise separable convolutions (used in MobileNet) to reduce CNN computation on mobile devices; graph convolution networks that extend convolution to irregular, non-Euclidean data such as social graphs and molecular structures; learnable or adaptive filter banks that change kernel shape during inference; and quantum convolution algorithms that may offer super-polynomial speedups over classical FFT for certain problem sizes. In signal processing, sparse-approximation methods (compressed sensing) exploit convolution structure to recover signals from far fewer measurements than the Nyquist rate traditionally demands.