I hope we'll be able to do better once we have an open-source storage backend that stores contents in a central database (like we do with Jujutsu at Google).
Note that the solution probably involve a local daemon process for caching reads and optimistically accepting writes and then sending them to the server in the background. That's what we do for latency at Google. But that's just an optimization and can come after the server is done.
If you're interested in working on it, it would help if you joined our Discord server.