How to Optimize Scaled Dot Product Attention?

yash-sri.xyz

2 pointsyash-sri191 comment

In this worklog, I explore how to optimize SDPA across cache, quantization and device specific optimizations.