# LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time

**Authors:** Teresa Delgado, Hugo Hazard, Martin Benfeghoul, Jun Wang, Zafeirios Fountas  
**Lab:** University College London · **Published:** 4 Nov 2026 · **Upvotes:** 34 · **Level:** Intermediate  
**Links:** [arXiv](https://arxiv.org/abs/2611.07025) · [PDF](https://arxiv.org/pdf/2611.07025) · [Code](https://github.com/lowram-model/lowram) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2611.07025.html)

## Curator's notes

**TL;DR:** LowRAM compresses KV caches by predicting values from keys and low-rank key compression, achieving up to 5× faster decoding at 4× compression.

- **What's new:** Introduces per-head value prediction from keys and cross-layer low-rank key compression, a combination not used in prior KV cache compression methods.
- **Key result:** LowRAM reaches 26.9× compression with 94.5% of baseline performance on long contexts.
- **Why it matters:** Builders can run larger language models with less memory, enabling higher throughput and batch sizes.

## Key numbers

- **94.5%**: reaching 26.9times compression at 94.5% of baseline performance on long contexts

## Abstract

Autoregressive decoding relies on the KV cache, whose memory footprint grows linearly with context length and caps batch size and throughput. Existing post-training methods exploit redundancy across tokens, layers, and feature dimensions, or lower numerical precision. We identify a complementary axis: within each attention head, values are largely predictable from their corresponding keys. We exploit this with value cache amortisation, which fits a lightweight per-head predictor at test time and stores residuals only for the tokens it reconstructs worst, giving 4times near-lossless value compression. The predictor can be initialised by meta-learning for substantially reduced prefill overhead, or fitted from scratch in settings that allow a one-off cost, such as multi-query inference. We pair value amortisation with cross-layer low-rank key compression to produce LowRAM, an asymmetric compression pipeline that stays close to lossless across three model families. By reconstructing only a selected subset of KV pairs at each decoding step, LowRAM can keep decoding overhead minimal and achieve up to 5times higher peak decode throughput at 4times compression. LowRAM further composes with quantisation and eviction, reaching 26.9times compression at 94.5% of baseline performance on long contexts. These results show that treating keys and values as structurally distinct enables efficient, aggressive, and composable KV cache compression.

## What you could build

1. Implement a KV cache wrapper that applies LowRAM compression for open-source LLM inference.
2. Create a benchmarking tool comparing throughput with and without LowRAM on various context lengths.
3. Develop a plugin for popular inference frameworks to auto‑select KV pairs for reconstruction.

## Quiz yourself

What two main techniques does LowRAM combine to compress the KV cache?

**Answer:** It combines value cache amortisation (predicting values from keys) and cross-layer low-rank key compression.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.