# CompressionBench: How can your coding agents compress LLMs?

**Authors:** Xinrui Zhong, Zhifeng Li, Zhiming Wang, Yongqi Liu, Shanglin Yang, Yang Zhou, Zikun Li, Zhuoming Chen, Beidi Chen  
**Lab:** Carnegie Mellon · **Published:** 4 Nov 2026  
**Links:** [arXiv](https://arxiv.org/abs/2611.06858) · [PDF](https://arxiv.org/pdf/2611.06858) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2611.06858.html)

## Key numbers

- **3.4%**: …lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream…
- **77.33%**: …agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy
- **79.56%**: compared with 79.56% for BF16

## Abstract

Recursive self-improvement requires AI systems to improve the models and infrastructure used by subsequent systems. LLM compression offers a concrete efficiency objective: retain a pretrained model's behavior within a smaller model storage budget. We study whether coding agents can carry out this step autonomously. We formulate compression as a terminal task in which an agent receives a source checkpoint, storage limit, inference environment, and time budget, then constructs compressed weights and executable inference code. The agent controls its algorithm and experiments; perplexity feedback guides search, and independent verification checks the final model. Across six agents and six model--size settings, 29 of 36 submissions pass verification. Selecting the lowest-perplexity agent design per task yields an average 3.4% perplexity increase over BF16 and 77.33% downstream accuracy, compared with 79.56% for BF16. Recorded experiments reveal how agents achieve these outcomes and where improvement breaks down. Agents lower PPL in 21 of 24 revisions at nearly fixed storage; complete-model feedback reverses 17 of 79 prior local rankings; and generation and inference tests expose failures that likelihood alone misses. The results identify representation design, experimental judgment, and capability validation as central to agent-driven model compression, a measurable component of efficiency-oriented self-improvement.

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.