I always dream about this setup. But an unreliable network prevents it. mosh and VSCode Remote partially solved the problem of latency. There's is still a problem of unavailable network - when you are in flight.

nix solved my problem by and large for well modularized projects. Because, nix can provide nearly identical dev environment in practice.