TurboQuant: A First-Principles Walkthrough

TurboQuant is a vector quantization technique that compresses high-dimensional vector components in language models (KV caches, embeddings, attention keys) to 2–4 bits per coordinate while maintaining accuracy. The method uses random rotation to transform inputs into a known distribution, then applies a reusable codebook without requiring training or calibration. The article provides an interactive first-principles walkthrough with pedagogical demonstrations of the underlying mathematics.