Tag: pe

Impact of Positional Encoding: Clean and Adversarial Rademacher Complexity for Transformers under In-Context Regression arXiv:2512.09275v1 Announce Type: new Abstract: Positional encoding (PE) is a core architectural component of Transformers, yet its impact on the Transformer’s generalization and robustness remains unclear. In this work, we provide the first generalization analysis for a single-layer Transformer under in-context…