分块 KV-Cache 压缩存在周期性弱点：相位敏感性问题研究

View PDF

Abstract:Chunked KV-cache compression reduces the memory and attention costs of long-context inference by compressing windows of consecutive tokens into fewer cache entries at a fixed stride. Such compression also introduces a new positional coordinate: a token's phase, or its position relative to compression-window boundaries. We uncover a systematic asymmetry in models using such compression: the same information can be easy to retrieve at one phase and difficult at another. We call this periodic variation in retrieval performance phase sensitivity. In large open-weight models with such compression, long-context retrieval accuracy can differ by up to 40 percentage points across phases, revealing periodic weak spots that average benchmark scores can conceal.
To investigate this behavior, we pretrain a family of transformers from scratch across multiple KV-compression designs, reproducing phase sensitivity across the variants. Mechanistic analysis using causal interventions in these models reveals phase specialization: different attention components contribute asymmetrically to retrieving information at different source phases. We further analyze idealized retrieval models, showing how gradient flow dynamics may favor sharp phase specialization. Evaluating models with chunked KV-cache compression thus requires measuring across compression phases: high average accuracy can coexist with systematic positional failures.

Comments:	65 pages, 20 figures, pre-print
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
ACM classes:	I.2.6; I.2.7
Cite as:	arXiv:2609.36322 [cs.LG]
	(or arXiv:2609.36322v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2609.36322 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Xingyu Zhu [view email]
[v1] Mon, 28 Sep 2026 21:54:14 UTC (1,340 KB)