Transformers, Time Series, and the Myth of Permutation Invariance

There’s a common misconception in ML/DL that Transformers shouldn’t be used for forecasting because attention is permutation-invariant.

Latest evidence shows the opposite, such as Google’s latest model, where the experiments show the model performs just as well with or without positional embeddings.

You can find an analysis on tis topic here.