/ Paper arXiv 2611.13548
In short
In shortReport a mistake
Key numbers
From the paperCopied from the abstract, not written by AI.
Why this ranked
MeasuredPublic numbers, not AIHow ranking works
5 / 100#138 of 1200 papers that day, from GitHub, Hugging Face, Hacker News and Semantic Scholar, looked up the morning it was ranked.
Builder evidence0 / 40
- No public code found
Community attention0 / 30
- ▲ 0 upvotes on Hugging Face
Author track record5 / 15
- From Hugging Face (a known lab)
What the paper shows0 / 15
- Nothing found
Abstract
From the paperThe authors' own words.
Sparse autoencoders (SAEs) are commonly evaluated by variance explained and ablation damage. We show how massive activations undermine both evaluations, and propose a correction. Massive activations cause up to 99.9% of activation variance to lie along the first principal component (PC1). Because the PC1 component has no unique sparse decomposition, each SAE run rebuilds it from a different set of features. These features dominate importance rankings: an average of 47 of the 50 most important features are unmatched in a separate run, compared with about 27 of a random 50. A dictionary can also report 99.96% of variance explained while leaving 33% of the variance outside PC1 unexplained. We bypass PC1 by training the SAE on activations with PC1 removed, then adding PC1 to the reconstruction. In affected models, the top-50 feature sharing rate rises from 4-39.5% to 73-99.5%. At the same TopK sparsity, the resulting SAE degrades the model up to 44% less and reduces content reconstruction error by up to 33.9%.
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.