For those who don't know the term "kernel smoothing", it just means

    ∑ᵢ yᵢ · K(xᵢ, xₒ) ⁄ (∑ⱼ K(xⱼ, xₒ))

In regular attention, we let K(xᵢ, xₒ) = exp(<xᵢ, xₒ>).

Note that in Attention we use K(qᵢ, kₒ) where the q (query) and k (key) vectors are not the same.

Unless you define K(xᵢ, xₒ) = exp(<W_q xᵢ, W_k xₒ>) as you do in self-attention.

There are also some attention mechanisms that don't use the normalization term, (∑ⱼ K(xⱼ, xₒ)), but most do.