It seems like attention methods are still evolving, just like residual connections. I’m sure there is a lot left to discover too.