Google's TurboQuant compression algorithm reduces LLM memory usage 6X via compression due to random data vector rotation

https://mashable.com/article/google-ai-compression

"alleviates key-value cache/ vector search bottlenecks, 2 primary operational phases AI models frequent to retrieve/ match high-dimensional data vectors... enables key-value pair compression... operates out of the box on existing Gemma, Mistral or similar architectures... optimizes data center capacity, offering a vital computational solution to help curb the massive energy and infrastructure demands of modern artificial intelligence ... reduced: RAM requirements, water for cooling, energy use"