Embedding Capacity Explorer — How Much Knowledge Fits in a Vector Space

Knowledge Base

Embedding dimension d 24

Concepts stored N 40

Collision threshold τ 0.40

Live Readouts

Mean cos similarity

0.00

Max cos similarity

0.00

Colliding pairs

0 / 0

Bound √(2·lnN/d)

0.00

Well under capacity — concepts stay nearly orthogonal.

How it works

Every "concept" here is a random unit vector in R^d — a stand-in for one entry of an embedding table (a word, a fact, a document chunk). Two truly unrelated concepts should sit near-perpendicular: cosine similarity ≈ 0.

unit vectors u,v ∈ R^d, random direction
E[cos(u,v)] = 0,  stdev ≈ 1/√d
over C(N,2) pairs, the expected MAXIMUM similarity
grows like:  max cos ≈ √(2·ln N / d)

That bound is the point: a fixed-dimension embedding space cannot hold an unlimited amount of "universal knowledge" while keeping every concept distinguishable. Pack in more concepts N, or shrink the dimension d, and random directions start pointing the same way — a collision, i.e. two unrelated concepts the model can no longer tell apart.

d slider — the capacity of the space (how many numbers describe one concept).
N slider — how many concepts are packed in at once.
τ (threshold) — how similar is "too similar"; pairs above it are drawn as red links and counted as collisions.
Grow — adds concepts one at a time so you can watch collisions emerge live, exactly as the √(2 ln N / d) curve predicts.

This is the real mechanism (concentration of measure / the Johnson–Lindenstrauss regime) behind why word2vec, sentence embeddings, and every RAG vector database pick a dimension in the hundreds: too low, and unrelated facts blur together; unnecessarily high, and you waste memory and search speed for no extra distinguishing power.

Concept (embedding vector) Collision (cos sim > τ)

⚠ Couldn't load the 3D engineThree.js failed to load from the CDN. Check your connection and reload.