A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention

February 06, 2024

https://arxiv.org/pdf/2402.03902