"...
we propose a
Linear Attention Mechanism which is approximate to dot-product
attention with much less memory and computational costs. The
efficient design makes the incorporation between attention
mechanisms and neural networks more flexible and versatile.
..."