# TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization

**Authors:** Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng  
**Lab:** University of North Texas · **Published:** 4 Nov 2026  
**Links:** [arXiv](https://arxiv.org/abs/2611.07217) · [PDF](https://arxiv.org/pdf/2611.07217) · [Code](https://github.com/HanzhiZhang-Ulrica/TileQuant) · [This page](https://omen-bit.github.io/ai-paper-daily/paper/2611.07217.html)

## Key numbers

- **2.66×**: Experiments show that TileQuant reaches 2.66x the original model's generation throughput
- **16-bit**: the highest-scoring tiles remain at 16-bit and the rest use 8-bit
- **8-bit**: the highest-scoring tiles remain at 16-bit and the rest use 8-bit

## Abstract

Post-training quantization (PTQ) reduces large language model (LLM) deployment costs, but preserving quality requires accounting for uneven weight sensitivity. Uniform PTQ uses one weight bit-width, importance-aware PTQ adjusts scales or reconstructs weights within that width, and column-wise mixed precision retains selected columns at higher precision. However, sensitivity varies within columns, so whole-column retention also protects less sensitive weights, while runtime outlier detection adds decoding overhead. We therefore introduce TileQuant, a training-free framework that assigns precision to hardware-aligned 2D weight tiles. Offline calibration provides input-activation-aware scoring from forward statistics for lower calibration cost and gradient-informed scoring from loss sensitivity for stronger quality preservation. Within each projection's retention budget, the highest-scoring tiles remain at 16-bit and the rest use 8-bit. The resulting map guides reconstruction into compact weight buffers, which the kernel reuses for native integer and 16-bit computation with local accumulation to reduce memory traffic. Experiments show that TileQuant reaches 2.66x the original model's generation throughput. Across language modeling, reasoning, long-context retrieval, and question answering, it preserves quality and surpasses quantized and original models on multiple tasks. These results support selecting precision units that capture local sensitivity while enabling efficient hardware execution. The implementation is: https://github.com/HanzhiZhang-Ulrica/TileQuant

---

Notes written by AI from the paper, numbers copied from the abstract. Check the paper before you rely on them. From AI Papers Daily.