Self-attention is required. The model must contain at least one self-attention layer. This is the defining feature of a transformer — without it, you have an MLP or RNN, not a transformer.
人 民 网 版 权 所 有 ，未 经 书 面 授 权 禁 止 使 用
。同城约会对此有专业解读
Александра Синицына (Ночной линейный редактор)
Cheyenne MacDonald for Engadget