LDA Topic Model: Dirichlet Mixtures of Document Topics (2D Simplex)

Write documents the way LDA assumes: draw a topic mixture from a Dirichlet, then tokens from topics. Watch how alpha controls peaky vs blended documents.

Switch to 3D
Controls

Presets

Parameters

Model checks

The science

Generative story (LDA). Latent Dirichlet Allocation assumes that every document is a mixture of topics and every topic is a distribution over words. To write a document: draw the mixture θ ~ Dirichlet(α, ..., α); for each token draw a topic zi ~ Categorical(θ), then a word wi ~ φzi.

The concentration α. Small α (< 1) puts θ near the corners of the simplex: documents lean on one or two topics. α = 1 is uniform on the simplex; large α makes every document a near-even blend. Exact moments used by the checks: E[θk] = 1/K, Var = (K-1)/(K²(Kα+1)), Cov(θi,θj) = -1/(K²(Kα+1)); the marginal of one component is Beta(α, (K-1)α); two tokens of one document share a topic with probability (α+1)/(Kα+1).

Sampling. Dirichlet draws use normalised Gamma(α) variates (Marsaglia-Tsang, with the boost for α < 1). The Kolmogorov-Smirnov check is done in logit space with log-gamma variates because at very small α the weight θ1 rounds to exactly 0 or 1 in double precision.

Simplifications. This page only generates documents: the topic-word distributions φ are fixed toy lists of 10 words per topic, uniform inside a topic. Real topic modelling runs the story backwards (collapsed Gibbs sampling or variational inference) to infer θ and φ from a corpus. The simplex plot shows the first two topics and lumps the rest into the third corner.

-
-