Residual Vector Quantization: How AI Generates Music Token by Token

AI music tools like MusicGen and Suno don't generate a waveform in one shot — they compress audio into continuous embeddings, quantize each one into a handful of discrete tokens through a cascade of residual codebooks (coarse-to-fine), and then have a transformer predict that token stream autoregressively, one time step after another. This simulator renders the whole pipeline: a target embedding snapping to the nearest codeword at each RVQ level, the leftover residual falling through to the next, finer codebook, and the resulting tokens filling in a scrolling piano-roll grid exactly as a real audio-token model would generate it. Adjust the codebook depth, size and sampling temperature and watch the residual error, perplexity, and effective bitrate respond live.