🧭 Word Embeddings and the Semantic Vector Space

About this simulation

A word embedding is a list of numbers — a vector — that a machine learning model assigns to a word so that words used in similar contexts end up close together in space. Real systems like word2vec or GloVe learn vectors with hundreds of dimensions from billions of words of text; this simulation uses a small, hand-placed 3D space of 34 words so the geometry stays visible and explorable.

How it works

• Each glowing point is a word; its color shows which conceptual category it belongs to.
• Words with related meanings are placed near each other, so clusters form naturally — animals in one region, emotions in another, and so on.
• Cosine similarity measures how closely two vectors point in the same direction, regardless of length; it is the standard way to compare word embeddings. Selecting a query word draws lines to its most similar neighbors and lists their similarity scores.
• Because meaning is encoded directionally, vector arithmetic works: subtracting the "man" vector from "king" and adding "woman" lands almost exactly on "queen" — the model has implicitly learned a "royal ↔ gender" direction.

Controls

• Category — filters and highlights one conceptual cluster of words.
• Query word — pick a word to see its top-5 nearest neighbors by cosine similarity.
• Training progress — 0% shows words scattered randomly (an untrained model); 100% shows the fully organized semantic space a trained model would produce.
• Similarity graph threshold + Show full similarity graph — draws a line between every pair of words whose cosine similarity exceeds the threshold, revealing cluster structure at a glance.
• ➕ Run analogy — animates the classic king − man + woman ≈ queen vector arithmetic and highlights the nearest resulting word.

Did you know?

This "vector arithmetic" property, popularized by the 2013 word2vec paper (Mikolov et al.), was one of the first striking pieces of evidence that neural networks could learn abstract relationships — like gender or capital-of — as consistent directions in space, not just memorized associations.