In most use cases I don't see the benefit over a RDP access to that remote server. With RDP you as well have control over all the over things that you might want there, starting with a proper Terminal, File Manager, Process Monitor etc.
Personally, I run my beefy Win/GPU/ML setup in the basement, while using it via RDP from my Mac. Awesome experience for me.