Flex Attention – How to Scale Attention Models to a Billion Users?

The title is a clickbait. I don’t actually know how to scale attention to serve billion users, but I feel Flex Attention is a great step in this direction.