Emergent Communication — Policy Graph (2D)

Language Parameters

Vocabulary size 4

Learning rate 0.15

Training speed 10 ep/s

Latest Round

Waiting for first episode…

Live Stats

Episodes

Success (last 50)

—

Protocol entropy

—

Best so far

How it works

This is the same Lewis signaling game that drives emergent-communication research in multi-agent RL, but drawn here as a live policy graph instead of a 3D scene. Each episode a random target concept is shown to the Sender, which samples a discrete symbol from its own learned distribution over the vocabulary:

P(symbol | concept) = softmax(L_sender[concept])

The symbol crosses the channel with no built-in meaning. The Receiver samples a guess from its own distribution over concepts, conditioned only on the symbol it heard:

P(guess | symbol) = softmax(L_receiver[symbol])

Reward is 1 if the guess matches the target, else 0. Both agents update with REINFORCE, nudging the logits of the action they actually took toward higher probability when the round succeeded, away from it when it failed:

advantage = reward − 0.5
L[state][action] += lr · advantage · ( 1[a=action] − P(a) )   for every action a

Every edge on the graph below is drawn with thickness and opacity proportional to the actual current probability in that concept's or symbol's softmax row — the whole policy table, not just one sampled path — so you watch the full distribution collapse from a haze of near-equal edges into one confident line per concept as a shared code is discovered. The strip charts track the rolling success rate and average protocol entropy across training, a purely 2D readout with no 3D equivalent.

Vocabulary size — fewer symbols than concepts forces the agents to reuse a symbol for more than one object, capping the achievable success rate (a pigeonhole limit visible directly as edges that never fully separate).
Learning rate — how aggressively logits move after each round; too high causes the graph to thrash instead of converging to clean lines.
Training speed — episodes simulated per second; the graph still replays a sampled recent round's animated token so you can watch individual transactions even at high speed.

Concept → symbol edge (sender policy) Symbol → concept edge (receiver policy) Correct round Wrong round