🌐 Translation Quality Estimator — Embedding Alignment Live

Translation quality estimation predicts how good a machine translation is without access to a human reference translation. Real QE systems (like COMET-QE or OpenKiwi) feed the source sentence and the machine translation through a multilingual encoder and compare the resulting embeddings — sentences that mean the same thing land close together in the shared embedding space, while mistranslations drift apart. This simulation builds a small, transparent version of that same idea: synthetic word embeddings, a shared cross-lingual concept space, and a real cosine-similarity score.

Every word concept gets a deterministic 20-dimensional vector generated from a seeded hash of its name. A source-language word and its correct translation are both built by adding independent small noise to that same shared concept vector — mimicking how real cross-lingual embedding alignment (e.g. MUSE, LASER, or a bilingual dictionary-induced mapping) places true translation pairs near each other despite being different surface words. An unrelated distractor word gets its own independent vector, so substituting the wrong word makes the translation vector drift away from the source.

Each sentence vector is a weighted average of its word vectors (80%) and its adjacent word-pair, or bigram, vectors (20%). Averaging word vectors is the classic bag-of-embeddings sentence representation; adding bigram vectors gives the representation some sensitivity to word order, since reordering words changes which bigrams appear even though the same words are still present. This is the same trade-off real fastText- or SIF-style sentence embeddings make between simplicity and order sensitivity.

The quality score is the cosine similarity between the source sentence vector and the translation sentence vector, rescaled from its typical −0.5…0.9 range onto a 0–100 scale. Cosine similarity measures the angle between two vectors regardless of their length, which is exactly what embedding-based QE metrics use to compare meaning while ignoring superficial scale differences. A faithful translation keeps this angle small (high score); substituting, omitting, or reordering enough words widens the angle and the score falls in real time as you move the drift slider.

Substituting a word swaps its vector for an unrelated one, pulling the average sharply away from the source — a few substitutions already show a large drop. Omitting words shrinks the sentence average toward whatever remains, which also degrades the score once enough content is missing, but a single missing function-like word matters less. Reordering keeps every word's vector in the sentence average unchanged, so it only shows up through the bigram component — a real reflection of why bag-of-words-style metrics are less sensitive to word order than to actual mistranslation of content.

The top panel takes every word vector in the current source and translation sentences (all 20-dimensional) and projects them down to the 2 directions of greatest variance using principal component analysis, computed live via power iteration on the covariance matrix. This is the same dimensionality-reduction technique used to visualise real word and sentence embeddings (e.g. plotting BERT or word2vec vectors). The two larger dots are the sentence centroids; the dashed line between them is the visual analogue of the alignment distance shown in the stats panel.

The core mechanism — comparing source and translation embeddings with cosine similarity in a shared cross-lingual space — is genuinely how reference-free QE and cross-lingual retrieval systems operate. What's simplified here is the embedding itself: production systems use deep multilingual transformer encoders trained on billions of words, while this simulation uses small deterministic pseudo-random vectors so the whole pipeline stays inspectable and reproducible in the browser. The mathematics of alignment, averaging, and cosine comparison, however, are the real thing.