Graph Attention Networks: Learned Attention Visualizer

Graph & Query

Topology

Query node #0

Attention Mechanism

Heads 2

Active head 0

LeakyReLU slope α 0.20

Softmax sharpness 1.5

Live Attention Readout

Σ α (sanity check)1.000

Entropy H(α)0.00 nats

Top neighbor—

How it works

Graph Attention Networks (Veličković et al., 2018) replace a fixed averaging rule with a learned weight on every edge, so a node can pay more attention to some neighbors than others instead of treating them equally.

Wh_i = W · h_i                          (linear projection, per head)
e_ij = LeakyReLU( aᵀ [Wh_i ‖ Wh_j], slope )   (unnormalized attention score)
α_ij = softmax_j(e_ij)                  (normalize over neighbors j ∈ N(i) ∪ {i})
h_i' = σ( Σ_j α_ij · Wh_j )              (weighted aggregation → new embedding)

Query node — the node whose incoming attention is visualized; every edge into it is drawn brighter and thicker in proportion to its α_ij, all other graph edges stay dim.
Heads — GAT runs several independent (W, a) pairs in parallel ("multi-head attention") and concatenates or averages their outputs; switch the active head to see how each one learns a different weighting of the same neighbors.
LeakyReLU slope — the negative-side slope of the scoring nonlinearity; at 0 it degenerates to a plain ReLU gate, so some raw scores can be clipped to a flat 0 before the softmax.
Softmax sharpness — a temperature-like multiplier on the raw scores before the softmax: low values spread attention almost uniformly across neighbors, high values concentrate nearly all of it on a single dominant neighbor.
Σ α and entropy — the softmax guarantees the weights on a node's neighborhood always sum to exactly 1; entropy H(α) = −Σ α·ln α measures how peaked (low H) or spread out (high H, up to ln(degree+1) for a uniform split) that distribution is.

Real-world relevance: this exact mechanism underlies GAT-based recommendation systems, molecular property prediction, and traffic-forecasting graphs, where not every neighboring node, atom, or road segment deserves equal say in an update.

Query node High-attention neighbor Low-attention / other edges

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.