When Covid hit and my main work machine was still a desktop, I worked from my home using VS Code and the remote plugin. This lasted for 18 months, and the only time I ever had an issue was due to a power outage in the office. Data, code (everything, really) lived on my work desktop that was sat in an empty office while I was at home.
There's no latency, the terminal opens as if you were on the remote machine, and ports are automatically forwarded. It runs code, tests, etc. all on the remote machine.
If there's a better solution out there it's almost certainly down to IDE preference.