I remember the original twitter thread on the topic but haven't heard much about it since then. It would be interesting to see a nice arxiv submissions with some initial experiments across common tasks and architectures. Would also be interested in hardware efficient versions for fixed point data.

Here is some background and more thoughts/references on this on this topic: https://github.com/brohrer/sharpened-cosine-similarity