HuggingFace Daily Papers（社区热门论文）· HuggingFace Daily Papers（社区热门论文）·· 6 天前AI 评分40
自回归 Transformer 如何从局部观测外推混沌系统的全局动力学
AI 导读
小型自回归 Transformer 仅用受限参数区间采样的轨迹训练，就能在训练分布之外的参数上闭环复现倍周期分岔、混沌动力学与吸引子结构。在 logistic 映射上，模型重现了直至周期 128 的连续倍周期分岔，得到有限阶标度比 4.6687，与 Feigenbaum 常数误差在 5×10⁻⁴ 以内。因果干预显示，控制参数信息经注意力机制进入状态预测，并塑造闭环动力学。
正文
Abstract:Autoregressive models are trained to predict a system's behavior one step at a time, and recursive generation allows the learned dynamics to unfold over long horizons. To what extent can such dynamics learned from local observations recover broader organization of an underlying system that was only partially observed during training? Here we study small autoregressive transformers trained from scratch on trajectories sampled from restricted parameter regimes of several non-linear dynamical systems, including logistic and sine maps, the Lorenz system, and the generalized Hopf system, with control parameters and state trajectories represented as sequences of continuous tokens. Under closed-loop evaluation at parameters far outside the training distribution, the models can recover self-similar period-doubling cascades, chaotic dynamics, and attractor structures with remarkable visual and numerical fidelity. For the logistic map, a transformer reproduces successive period doublings up to period 128, yielding a finite-order scaling ratio of 4.6687, matching the Feigenbaum constant to within $5\times10^{-4}$. We further investigate how these structures emerge over the course of training, and reveal with causal interventions how control-parameter information is processed through attention into state prediction and shapes the resulting closed-loop dynamics. These results suggest that a surprisingly narrow window into a system's local behavior may suffice for autoregressive transformers to generalize to its unseen global dynamical organization.
| Subjects: | Machine Learning (cs.LG); Chaotic Dynamics (nlin.CD) |
| Cite as: | arXiv:2609.38814 [cs.LG] |
| (or arXiv:2609.38814v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.38814 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yilun Liu [view email]
[v1]
Wed, 30 Sep 2026 02:42:58 UTC (6,418 KB)
来源：HuggingFace Daily Papers（社区热门论文） · arxiv.org