# Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

**Authors:** Moritz Thoma, Maximilian Groezinger, Maximilian Forstenhäusler, Emad Aghajanzadeh, Manoj Rohit Vemparala, Christos Anagnostopoulos, Pierpaolo Mori, Nael Fasfous, Alexander Frickenstein, Daniel Mueller-Gritschneder, Ulf Schlichtmann  
**Lab:** Technical University of Munich · **Published:** 22 Oct 2026 · **Upvotes:** 4  
**Links:** [arXiv](https://arxiv.org/abs/2610.32878) · [PDF](https://arxiv.org/pdf/2610.32878) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2610.32878.html)

## Key numbers

- **15%**: while KFAC-SVD achieves an average perplexity improvement of 15%
- **70B**: …zero-shot accuracy gains of up to 4.8 p.p. that generalize to model sizes of 70B parameters

## Abstract

Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of LLMs. However, existing methods face two key limitations: (i) global rank allocation, where uncalibrated error proxies fail to capture complex error propagation, and (ii) decomposition quality, where Fisher-based estimators suffer from severe rank collapse. In this work, we address these limitations by introducing Layer-wise Error Modeling Search (LEMS) and KFAC-SVD. LEMS advances rank allocation by introducing a layer-wise error surrogate that integrates local and global layer importance alongside a propagation bias, enabling effective global rank allocation via an ILP formulation. KFAC-SVD improves decomposition quality by utilizing token-wise statistics, mitigating the rank deficiency observed in prior Fisher-based SVD approaches. Across Mistral, Qwen3, and Llama 3 model families, we show that LEMS consistently outperforms existing search strategies, delivering significant zero-shot accuracy gains of up to 4.8 p.p. that generalize to model sizes of 70B parameters, while KFAC-SVD achieves an average perplexity improvement of 15%. Project page and code: https://lems-svd.github.io

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.