Capstractcscapstract:2611.00038

Computer Science > Computation and Language

Title:Attentional Dysregulation and Gain Modulation in Long-Context Language Models

Abstract:

Language models sometimes abandon an active instruction while its text remains in the key–value cache. To distinguish this behavior from failures of reading or recall, we develop an 18-item transcript instrument and validate it against delayed rule use in independent tasks. A correlated two-factor model separates maintenance failures from premature transitions, and the total score achieves an inter-rater intraclass correlation of 0.89. We then test a selective attention-logit gain intervention in a preregistered, placebo-controlled study of three frozen decoders. At the prespecified dimensionless dose of 0.4, exact joint completion increases from 60.2% to 68.1% in screen-positive runs (paired difference 7.9 percentage points, 95% CI 5.9–9.9) and by 1.1 points in screen-negative runs (95% CI −0.6–2.8). Higher gain reduces entropy further but compromises uptake of valid revisions. In repeated sessions the benefit declines, and the first withdrawal estimate falls below continued placebo, with an interval that includes zero. A fused attention kernel measures these changes without materializing the attention matrix. The response depends on the initial behavioral profile, the distribution of attention across heads, and exposure history.

Comments:16 pages, 4 figures, 6 tables
Subjects:Computation and Language (cs.CL); Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:capstract:2611.00038 [cs.CL]
 (or capstract:2611.00038v1 [cs.CL] for this version)

Submission history

From: Mira V. Iskandar
[v1] Thu, 19 Nov 2026 04:12:06 UTC (245 KB)

Current browse context:

cs.CL
Change to browse by:

Export citation