There is already another breakthrough on top of RoPE that promises even better scaling to larger contexts (based on NTK-awareness) - see my submission here:
https://news.ycombinator.com/item?id=36533816
The lmsys team is already looking into it, expect further progress soon.
It's really amazing to follow this progress in realtime, even if the pace can sometimes feel exhausting!