A retnet commit has now appeared in Microsoft's torchscale repo:

https://github.com/microsoft/torchscale/commit/bf65397b26469...