Google unveils TurboQuant to reduce AI memory use

Google Research has announced TurboQuant, a memory compression method it says can shrink the KV cache used during AI inference without reducing model accuracy.

The researchers said TurboQuant combines a vector quantization approach called PolarQuant with a training and optimization technique called QJL to reduce cache bottlenecks.

Google Research said the approach can cut inference working memory by at least 6x, which could lower the cost of running AI models if it works outside the lab.

The company has not deployed TurboQuant broadly, and the method targets inference memory rather than training, which still requires large amounts of RAM.

🔗 Source: TechCrunch

🧠 Food for thought

Implications, context, and why it matters.

Google’s new method uses polar coordinates to fix a compression flaw

Common compression can take a “memory tax” since it must save extra metadata called quantization constants, which can add 1–2 bits per number and cut into the space savings 1, 2.
TurboQuant sidesteps that overhead by starting with PolarQuant, which converts vectors from standard grid coordinates into polar coordinates defined by a radius plus angles 3, 1.
The angles tend to cluster in a predictable way, so the method can skip storing those constants and shrink the metadata burden 3, 1.
Next, QJL compresses each transformed vector value into a single sign bit (+1 or -1), then uses an estimator that mixes a high-precision query with low-precision stored data to compute attention scores 3.

Open release draws developers and leaves market effects unclear

Google Research published TurboQuant with related work, then said it plans to present the results at ICLR 2026 next month 3.
VentureBeat wrote that developers began adapting TurboQuant for local AI tools like MLX (a machine-learning framework for Apple Silicon Macs) and llama.cpp (a widely used open-source runtime for running Llama-family models locally) within 24 hours of the announcement 1.
VentureBeat reported that shares of memory suppliers such as Micron and Western Digital moved down after the news, though Google Research did not back that link in its own post 1.
VentureBeat added that some investors may expect a sixfold drop in AI inference memory needs to soften demand for High Bandwidth Memory (HBM), which remains speculative and attribution-dependent 1.