Attention is all you need
The Transformer is a sequence-to-sequence (transduction) model that replaces sequence-aligned RNNs and convolutions with attention mechanisms. It uses self-attention to relate positions within a sequence and cross-attention to connect input and output sequences, enabling direct modeling of long-range dependencies and highly parallel computation.
