Every architecture stacks the same primitive — a weighted sum followed by a nonlinearity — but wires it differently. A perceptron is one such unit; an MLP stacks fully-connected layers of them; a CNN reuses one small set of weights across a sliding window; a Transformer replaces fixed wiring with content-based attention.
MLP: y = f(Wx + b)
CNN: y[i] = f(sum_k w[k]·x[i+k] + b) (shared kernel)
Attention: Attention(Q,K,V) = softmax(QK^T/sqrt(d))V
- Architecture — rewires the same node count into a different connection pattern; Transformer additionally shows attention edges (token ↔ token) instead of only forward layer links.
- Speed — how fast the forward-pass pulse travels through the graph.
- Hidden layers — depth of the MLP/CNN stack (Transformer keeps a fixed 4-token attention block).
- Replay pass — restarts the animated activation wave from the input layer.
Real systems pick architecture by data shape: CNNs exploit spatial locality (images), Transformers exploit long-range, content-based relationships (language) — which is why large language models moved from RNN/CNN stacks to attention.