/ Paper arXiv 2611.14209
In short
In shortReport a mistake
Key numbers
From the paperCopied from the abstract, not written by AI.
Abstract
From the paperThe authors' own words.
During extended wakefulness, mammalian neocortical networks experience localized periods of transient neuronal silence termed off-states. Here, we examine how analogous spatiotemporally concentrated disruptions affect sequential reasoning in autoregressive Large Language Models (LLMs). Evaluating 250 paired derivations on the GSM8K benchmark across two 8B models (Qwen3-8B and LLaMA-3.1-8B-Instruct), we contrast clustered bursts of attention head masking against budget-matched diffuse i.i.d. dropout. Concentrated bursts induce substantial reasoning degradation: at onset parameter B=0.10 (scheduling ~ 5.7% of vulnerable head-token volume), Qwen3-8B accuracy drops from 91.2% under diffuse noise to 78.8% under burst dropout ($Δ= -12.4$ percentage points [pp], exact McNemar p = 5.5 × 10^-6), while LLaMA-3.1-8B-Instruct drops from 84.0% to 76.8% ($Δ= -7.2$ pp, $p = 0.0198$). Conversely, dispersing the identical cumulative budget uniformly across generation steps via stratified quintile matching yields substantially milder degradation (retaining 91.2% in Qwen and 84.0% in LLaMA). In an exploratory factorial dissociation ($N=100$), single-step dropout density is associated with an acute drop in a single schedule draw ($-10.0$ pp on Qwen, p_holm = 0.0425), while multi-token continuity exhibits an inconclusive compounding trend ($-7.0$ pp, p_holm = 0.1892). Error analysis reveals that Qwen failures reflect derivation errors within formatted responses (conditional accuracy drop of $-11.4$ pp), whereas in LLaMA 22.2% of discordant losses lack a final answer format line (conditional gap narrows to $-5.0$ pp). These findings demonstrate that autoregressive reasoning chains are particularly vulnerable to concentrated structural disruptions, providing empirical benchmarks for fault tolerance in transformer architectures.
From the full paper
From the paperCopied from the paper itself by our code.
Reading the paper on arXiv…
Ask this paper
/ Ask a questionAnswers come from the full paper and name the section
/ Your notes
Select text in the abstract to highlight it or add a note. They stay in this browser.