Zipf's Law: Word Frequency vs Rank

Synthetic Zipfian Corpus

Vocabulary size V 400

Zipf exponent s 1.00

Corpus size N (tokens) 20,000

Real Text Overlay

Live Fit — Synthetic

Fitted exponent ŝ

–

R² of fit

–

Rank-1 count

–

Distinct words hit

–

Live Fit — Real Text

Fitted exponent ŝ

–

R² of fit

–

How it works

Zipf's law says the frequency of the r-th most common word in a language falls off as a power law of its rank:

f(r) ≈ C / r^s
log f(r) = log C − s·log r

In log-log axes that is a straight line with slope −s. For natural language s is close to 1: the most frequent word ("the" in English) occurs roughly twice as often as the second-ranked word, three times as often as the third, and so on.

Synthetic corpus — this simulator builds a discrete probability table p(r) ∝ 1/r^s over a vocabulary of V "words" (normalised so it sums to 1), then draws N tokens from it by inverse-transform sampling. The teal bars are the resulting word counts, re-sorted by observed frequency (empirical rank), not by the true generating rank — showing that sampling noise still lands on the same log-log line.
Vocabulary size V and corpus size N control how far the tail extends and how noisy the low-frequency ranks look; a small N relative to V leaves many words unseen (0 count).
Exponent s steepens or flattens the curve — s → 0 approaches a uniform distribution, s ≳ 1.5 makes a handful of words dominate almost the whole corpus.
Real text overlay — the amber bars tokenize whatever text sits in the box, count word frequencies, and fit the same log-log line live. Real English text consistently lands near s ≈ 1, which is exactly why the demo defaults to on.
The red curve on each row is the fitted power law, drawn from the least-squares regression on (log rank, log frequency).

Real-world relevance: Zipf's law is why stop-word lists exist (a tiny set of words covers most of any corpus), why byte-pair/subword tokenizers spend their merge budget on the most frequent character pairs first, and why naive frequency-based compression schemes can get most of their gain from a handful of common tokens.

Synthetic Zipf corpus Real text overlay Fitted power-law curve

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.