FedAvg Client Drift: Federated Model Aggregation

Federation Round

Non-IID data skew 1.00

Local epochs E 5

Local learning rate η 0.15

Aggregation State

Round

Global error

—

Avg client drift

—

Contraction (1-2η)

—

Clients (n = sample count)

How it works

Federated Averaging (FedAvg, McMahan et al. 2017) trains one shared model across devices that never upload their raw data. Each round, every client i starts from the current global weights w_g, runs E local gradient steps on its own private objective L_i, and only the resulting weight vector is sent back:

w_i ← w_g − η · ∇L_i(w_i),  repeated E times   (local SGD)
w_g ← Σ_i (n_i / N) · w_i                     (FedAvg aggregation)

Here each client's private loss is modeled as a simple quadratic bowl L_i(w) = ‖w − c_i‖² centered on that client's own data-optimal weights c_i. When clients hold non-IID data, the c_i are spread apart (the "skew" slider) — so local training pulls each client's weights toward a different point before the server ever sees them. This is real client drift: more local epochs E, a larger learning rate η, or more skew all push clients further from w_g before averaging, which slows convergence and can bias FedAvg away from the true minimizer of the global objective (addressed in practice by methods like FedProx and SCAFFOLD).

The bowl-shaped surface is the global objective L(w) = Σ (n_i/N) L_i(w) — a weighted average of every client's bowl. Its true minimum is the sample-weighted mean of the client optima; switching to equal weighting moves that target, showing how FedAvg's n_i/N weighting (not just plain averaging) determines which clients dominate the aggregated model.

Non-IID skew — how far apart each client's local data optimum is from the others.
Local epochs E — gradient steps each client takes before reporting back; more steps means more communication efficiency but more drift.
Learning rate η — local step size; each local step contracts the distance to c_i by a factor (1 − 2η).
Sample-weighted vs equal — real FedAvg weights each client's update by n_i/N (its share of total training examples); equal weighting lets a tiny client outvote a huge one.

Idle

Global model w_g True weighted optimum Colored dots = 5 clients

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.