Retentive Network: A Successor to Transformer for Large Language Models

The repo the paper is pointing to indicates that the code will be released within ~1 week! If the sheer difference in VRAM requirements and latency holds up, this will seriously be a major breakthrough for LLM architectures!

Can't wait to try it out

Brief twitter thread with performance metrics. Looks big if results hold up.

https://twitter.com/arankomatsuzaki/status/16811139775001845...