- Much more limited IDE experience if you use any graphical IDE. I prefer Pycharm because it is far superior to pretty much anything else out there when working in Python + Pytorch. You need to use a remote desktop solution like VNC in that situation, which is not remotely close to being on-par with local code prototyping.
Jupyter notebooks I abhor because of how poor they are in relation to any decent IDE. VsCode is the only tool that has a great remote dev workflow but it just isn't near the functionality of Pycharm when you use the latter daily. You also don't directly get the ability to plot and visualize data in something like matplotlib when working remotely which is again an issue. I'll still use it if I have to when VNC isn't snappy enough
- If data security is a concern, everything needs to occur through a VPN which is another intermediate step in your workflow to get started every time you open up your laptop.
- If you have spotty internet access or are traveling, the remote dev workflow suffers immensely.
The other thing I should mention is that outside of the standard data exploration + model training/inference workflow, there are other use-cases where being able to prototype locally is very advantageous. I write a lot of our tooling and internal libraries (for example: a keras-like model training framework in Pytorch) and that involves a lot of pytorch code that references GPUs but testing and prototyping that could easily be done with small models on a laptop. Not being able to do that at all is really annoying and having a native IDE experience when writing a library (especially when you rely on several internally developed libraries) is very critical in my experience.
Again, all of these are not individual deal-breakers, but I feel the pain almost every day.
Not everything needs to be gpt3 scale.