Yes, and it’s not even research anymore! Qwen uses linear attention: https://sebastianraschka.com/llms-from-scratch/ch04/08_delta...