In this worklog, I explore how to optimize SDPA across cache, quantization and device specific optimizations.