Мережі на основі радіальної бази функції: навчання з наростами, а не плоскими поверхнями

Radial Basis Function Networks

A standard feedforward network builds its decision surface out of hyperplanes — each hidden unit's pre-activation is a dot product w·x, and a sigmoid or ReLU turns that into a soft or hard half-space boundary. A radial basis function network does something different: each hidden unit computes a distance from the input to a stored centre, and turns that distance into a bump that is largest at the centre and decays outward. The most common choice is the Gaussian:

φ_i(x) = exp( -‖x - c_i‖² / (2σ_i²) )

c_i = centre of basis function i (a point in input space)
σ_i = width — how fast the bump decays away from c_i
y(x) = Σ_i w_i · φ_i(x) + b     (output is a weighted sum of bumps)

Локальне проти глобального наближення

Ця локальність є визначальною рисою. Прихований нейрон у багатошарового перцептрона впливає на вихід всюди в межах одного боку від його гіперплощини, аж до нескінченності; функція бази Гауса впливає на вихід лише в межах власного центру та його вплив поступово зменшується до нуля через кілька ширини. Це робить мережі RBF локальними наближеннями: додавання навчальних даних далеко від існуючих центрів не порушує відповідність біля цих центрів, що є корисною властивістю, коли модель потрібно оновлювати поступово без перенавчання всього з нуля. Ціна — прокляття розмірності — кількість горбів, необхідних для покриття області, швидко зростає з виміром входу, оскільки фіксована ширина Гаусса охоплює лише невеликий гіпероб'єм у високих вимірах.

Two-stage training

Because the model is linear in the weights once the bumps are fixed, RBF networks are almost always trained in two decoupled stages rather than by full backpropagation. First, choose the centres c_i — commonly by running k-means clustering on the training inputs, so centres land where the data actually is, or in engineering contexts by simply picking a subset of training points. Second, with centres and widths fixed, computing the output weights is linear least squares: build the matrix Φ of φ_i(x_j) for every training point and basis function, and solve w = (ΦᵀΦ)⁻¹Φᵀy (or its ridge-regularised form) in one shot — no gradient descent, no local minima, a closed-form solution. Widths σ_i are usually set from the spacing between neighbouring centres, often as some multiple of the average distance to the nearest few centres, to avoid gaps or excessive overlap between bumps.

Why they are universal approximators

Given enough centres, an RBF network with Gaussian (or any of several other admissible) basis functions can approximate any continuous function on a compact domain to arbitrary accuracy — the same universal approximation guarantee that holds for sigmoidal MLPs, proved for RBF networks by Park and Sandberg in 1991. Intuitively this is not surprising: a sum of narrow enough Gaussians centred densely enough can reproduce any smooth bump shape, in the same way a Riemann sum of narrow rectangles reproduces any integrable function as the rectangles narrow.

The Power of Radial Basis Functions

Radial Basis Function (RBF) networks are a powerful tool for solving a wide range of problems, particularly those involving interpolation and approximation. Unlike traditional neural networks, RBF networks use radial basis functions as activation functions, which means that the output of each neuron depends only on the distance between its center and the input data point. This makes RBF networks very efficient at interpolating data, meaning they can accurately predict the value of a function at any given point.

The key advantage of RBF networks is their ability to represent complex functions with a relatively small number of neurons. This is because each neuron only needs to be connected to a few training points, and the radial basis functions allow the network to capture the local behavior of the function. This makes RBF networks particularly well-suited for problems where data is sparse or noisy.

RBF networks are used in a variety of applications, including pattern recognition, time series prediction, and control systems. They have also been shown to be effective in solving problems that are difficult for other types of neural networks, such as those with high dimensionality or non-linear relationships between the input and output variables.

Frequently asked questions

Спробуйте наживо

Усе, що вище, працює прямо у вашому браузері — відкрийте Radial Basis Function Network і змінюйте параметри під час роботи. Нічого не встановлюється, нічого не завантажується на сервер, уся модель живе в одній вкладці.