Graph Attention Network: Attention Matrix View

Attention Heads

Layer Parameters

Softmax temperature T 1.00

Feature noise σ 0.60

Readouts

Classification accuracy

—

Graph edges / avg degree

—

Class A Class B Class C

Ring = true label · Disc fill = GAT prediction · right panel = α_ij heatmap

Focused Node

Click a node (left) or a matrix row (right) to inspect its attention weights α_ij.

How it works

A Graph Attention Network layer learns, for every node i, how much to trust each neighbor j when building a new representation. For a shared weight matrix W and attention vector a per head:

e_ij = LeakyReLU( a·[ W h_i || W h_j ] )
α_ij = softmax_j( e_ij / T )   over j in N(i) ∪ {i}
h_i' = ELU( Σ_j α_ij · W h_j )

This 2D view keeps the exact same forward pass as the 3D version but replaces the orbiting camera with two flat, 2D-native readouts: a spring-embedded node graph (a 2D force layout, not a projected 3D one) and an N×N attention-weight heatmap grid — the row-by-row α_ij distribution that a 3D scene can only show one node at a time.

Attention heads — more heads give the layer more independent ways to weigh neighbors before concatenation.
Temperature T — divides e_ij before softmax; low T sharpens attention onto the single best-matching neighbor (heatmap row → one bright cell), high T spreads it near-uniformly (row → flat band).
Feature noise σ — perturbs each node's input features around its true class center, degrading how well attention (and the final classifier) can separate classes.
Click a node or a matrix row — highlights its edges by α_ij on the graph, brightens its row in the heatmap, and reports the entropy of its attention distribution (low entropy = focused on one neighbor, high = spread out).

This mirrors the original GAT architecture (Veličković et al., 2018) used for semi-supervised node classification on graphs like citation networks, where a node's class is inferred from a learned weighting of its neighbors rather than a fixed average.

Left: 2D spring-embedded graph · Right: α_ij attention matrix · click a node or matrix row