Ask HN: Best practices for editing remote code locally?
Working on a remote codebase from a local computer accessible via ssh, what’s the best/easiest way to write, edit and test code?
When Covid hit and my main work machine was still a desktop, I worked from my home using VS Code and the remote plugin. This lasted for 18 months, and the only time I ever had an issue was due to a power outage in the office. Data, code (everything, really) lived on my work desktop that was sat in an empty office while I was at home.
There's no latency, the terminal opens as if you were on the remote machine, and ports are automatically forwarded. It runs code, tests, etc. all on the remote machine.
If there's a better solution out there it's almost certainly down to IDE preference.
I have heard stories from coworkers about work being lost when editing is done remotely instead of locally, so never ventured that way.
This could be a variety of scenarios like working remote on bad ISPs or cellular or a crappy VPN but all that matters is when it acts up it will cause lag, or dropped connections depending on severity. Too many interruptions while trying to focus.
For this reason I keep everything local and use an rsync wrapper over my build system that streams output back. The added delay for each command disrupts workflow less than it happening during editing.
With my VPS from DigitalOcean the ping will be 170ms and then its no bueno.
If it must be done manually, can you just ssh/rdp directly into the testing box? The network where the testing box exists could use some tunneling protocol (stunnel, socks, wireguard, vpc peering...) to make it more accessible from your dev machine or your company VPN.
I would love to use my main mac + VS Code to edit code and dispatch commands like compile / run
Which doesn't starts up any server on the remote server.
> The Visual Studio Code Remote - SSH extension allows you to open a remote folder on any remote machine, virtual machine, or container with a running SSH server and take full advantage of VS Code's feature set. Once connected to a server, you can interact with files and folders anywhere on the remote filesystem.
Once connected, even terminal runs on remote host. Moreover you get to forward ports from remote host to local using VSCode UI.
Also you can use embeded terminal and type "code myfile.txt" and it will open the file.
If used with windows, it also automaticaly remap localhost to the machine if you are launching a local server with your terminal (can be annoying tho)
Basicaly it try to do everything to make you feel that you aren't on a remote ssh connexion and it work really well
I can also choose to temporarily vertically scale the remote host if I want to run single-node operations (ie Pandas).
Keep it traditional baby all you need is vim and tmux.
The initial learning curve for these sorts of cli apps is a bit steep, but it’s a lifelong skill and easily learnable for any competent developer.
Writing code is 1000x more difficult than learning vim philosophy and memorizing some key bindings.
Since I do all my editing in terminal, I just have a portable environment in a git repo. It has an install script that soft links all my scripts and config files to the appropriate locations. My work environment comes with me everywhere!
It’s lightweight, fast, and works on any machine. After years in VScode I got fed up with Microsoft lock in for things like syncing configs and remote editing extensions.
Your point about windows may be true, but I’ve yet to work somewhere that hasn’t let me use a MacBook.
Also I’m a pentester at work, so I don’t have to write production ready code…I can see why developers might want all the shiny features/integrations that come with a language-specific IDE.
To each their own!
I run a dev VM on any machine(s) I'm actively working on that all share the same setup - today, that's my Macbook and my desktop at home. They aren't necessarily in sync, but they do share the same dotfiles and versions of everything, so if I want to switch my dev work all I simply need to do is a `git pull` on whatever project I'm working on, and then everything is caught up between the two.
With this, I just `mosh dev` (and then straight into `tmux`) on whatever computer I'm running into and just use `vim` for everything. If I really need a graphical debugger I can always open the VM graphically, but that's so rare that it hardly feels worth mentioning.
What’s your VMM setup? I just edit right on the remote machine. Lately I’ve been playing around with lxd/lxc for persistent container dev environments. This way I can avoid the VM overhead and it “feels” the same…but I have yet to fully explore this approach.
I just run it right in VMware Fusion / Workstation, and it works great. Extremely reliable, very low overhead still. It just takes a lot of storage is all :) But everything is always reproducible given the nature of the VM, without needing to pass around snapshots or clones.
I've been using Nebula to be able to connect to my primary machine. Then I use mosh to connect, which works great when my laptop suspends, I move locations, etc... I just open up my laptop from the car while waiting for my daughter, over the cell phone, from bed, from the back yard if it's nice... Or my Chromebook. Or my backup chromebook.
Lately I've been using LunarVim, which has a lot of enhanced functionality via TreeSitter and LSP. I use oh-my-tmux for settings and a few plugins for doing keyboard-based selection/cut/paste.
LunarVim is just an opinionated neovim setup with all the bells and whistles. I got tired of pushing along my 20 year old vi configs and trying to get various advanced features working through various plugins, so I started evaluating just using something else, including spacevim, kakoune, onivim2.. Ended up sticking with LunarVim.
What I like about this setup over tmux is that I feels a bit more integrated into my desktop environment, and I don't need as much keystrokes as with tmux. Tmux has a nice integration mode I've been using in the past but it's only supported by a few terminal.
I work on a remote dev server for most of my stuff, by choice, which allows me to quickly pull images, dependencies and the rest regardless on the quality of the network I'm working from and I also get a small battery boost since most the of the heavy work is done on the server.
I still use kitty windows or just detach from tmux when I wanna do something that benefits from kittys beautiful display protocol.
That said, don't underestimate the upfront cost of something like this. Getting the right plugins + getting comfortable took me a considerable amount of time.
This is definitely true. I've been a terminal guy for many years so its easy for me to recommend while I already have the muscle memory. It might be an uncomfortable few weeks, but I still argue that learning a new language or framework is MUCH harder than learning keybindings, coreutils, and shell operators.
Even if you took years to get vim right, I still don't think it matches to the other 3 for programming.
What is your vim setup like?
The only place I use vim is for config editing on servers but not for programming.
Why do you care how long it takes to start up? It's either few seconds or 15 seconds a few times a day.
People usually have enough RAM with a machine bought in the last 5 years and I don't see any buttons that fill half the screen? Most of the screen estate is folder tree or the code.
Vim key bindings are everywhere it's not just for vim and I never code on phone.
Good for you that you're not a student having to use a cheap laptop for everything and sometimes not having access to it at the moment. Not everyone is so fortunate.
Look at the first screenshot in the official VSCode repository, for example: https://github.com/microsoft/vscode Only about 20% of the screen is taken up by code. That may still be somewhat usable if you keep it fullscreen, but I prefer to have a 50/50 split with the web browser.
Most Vim binding plugins are a complete joke. I've heard that Emacs gets it right, but that's not what we're talking about.
I forgot to mention, Vim can be used for every language, including very obscure ones. With JetBrains, you need a separate IDE for each language — if there even is one.
flexibility - vim integrates very nicely with my terminal-heavy workflow. Its extremely hackable. I can zip around and pipe contents between my shell, unix tools, and vim buffers. For example, I can run a shell command from nvim and have its output automatically pasted into my current vim buffer at the cursors location.
lightweight - It's a blazing fast tiny executable rather than a clunky resource hungry electron app.
keyboard-centric - I enjoy my desktop the most when I don't have to move my hands from the keyboard. This is really just a personal preference. Also the vi keybindings are ubiquitous in unix tools and even some gui/web apps!
> VS code doesn't let you sync configs
I thought it's just a JSON file.
> It's a blazing fast
This depends, if you make your vim full blown to match the feature (which is probably not even possible), vim tends to start up pretty slowly.
Vim key bindings are pretty much available on most editors out there.
And you'll be losing so much from the vast list of plugins available in vs code.
You can't autocomplete table names in SQL strings like in JetBrains, I've quit using vim 15 years ago as my programming editor.
At my day job and for my personal infrastructure, I routinely edit files across a wide variety of shuffling environments. I am a pentester by trade and also enjoy homelabbing, embedded systems, and working on my ADHD-fueled inventory of fun projects. I am not a dedicated software engineer that has to write polished production-quality enterprise applications. All my shit is held together by duct tape and bubble gum. I 100% understand why you'd want a beefy dedicated IDE for a product you're working on full time.
> I thought it's just a JSON file.
If VSCode has changed in the past couple years to json file that's great! But last time I tried I wasn't able to move around my config easily without signing into some web service.
> This depends, if you make your vim full blown to match the feature (which is probably not even possible), vim tends to start up pretty slowly.
Absolutely right! I keep my plugins minimal. You can do almost everything the plugins can with vanilla vim.
> Vim key bindings are pretty much available on most editors out there.
Also true! In my experience, its wonky and usually requires YET ANOTHER plugin. They're never at parity with all the vim actions.
> And you'll be losing so much from the vast list of plugins available in vs code.
> You can't autocomplete table names in SQL strings like in JetBrains, I've quit using vim 15 years ago as my programming editor.
That's also fair, but I don't use most of those plugins. Only copilot (also available in nvim) and remote editing really amazed me from VSC. Everything else I can just do myself by spawning a shell, using pipes/redirects, and unix tools. For example, I don't need GUI elements for git baked into my editor...I just use the cli.
You can definitely autocomplete SQL table names, but it would require some manual configuration. My autocomplete mostly pulls from strings in my open buffers.
I'm not arguing that any either approach is inherently better, just stating my preference. And for the original topic of remote editing ssh+vim is my preferred solution. Absolutely language-specific IDEs have major benefits over a general purpose editor.
There's also the FOSS philosophical argument but that's just a personal belief and really doesn't matter for all practical purposes.
Do you have a tutorial/resource on learning this?
> :read !{command}
so ":read !pwd" will paste your current directory into your open buffer.
Another cool thing...you can pipe cli output directly to nvim:
> curl example.com | nvim -
will open a new nvim instance with a buffer containing the nice, pretty printed html of that site!
(2) Switching to the mouse is annoying. Dealing with lots of windows is inefficient compared to kb shortcuts.
(3) If a large part of your workflow happens in a terminal environment, it feels natural to use it.
If you can’t install software on the remote (or you can’t install a GUI stack), read on:
- Most IDEs like Intellij can be configured to do all their execution on a remote over SSH. I worked this way using PyCharm for my student job at UC Berkeley in 2012 and it worked great.
- SSHFS is okay but not great because of slowness. It’s fine for editing but sucks for running interpreted code locally. So, try to do all execution on the remote.
- VS Code has handy support of editing on a remote without needing SSHFS
- if permitted, a two-way file sync solution like Unison or Mutagen gets you native file system performance on the local. We used Unison for local editing with remote execution for Airbnb’s Ruby codebase for several years (2015-2018+). But it’s more fiddly to work with than SSHFS.
$ time sha256sum file
c2359f8b63dfef9f50e62b2e7b4ef2613c290c3a1ffa135cfa255b14eafd3ffb file
real 0m5,205s
user 0m0,127s
sys 0m0,011s
$ time sha256sum file
c2359f8b63dfef9f50e62b2e7b4ef2613c290c3a1ffa135cfa255b14eafd3ffb file
real 0m0,221s
user 0m0,093s
sys 0m0,004s
Do a search on all files and you'll need a break.
Code parsing definitely can take forever using JetBrains.
My average project size ~12MiB - 19MiB when using vendor/ directory with Golang. It's less than a minute wait on a 1MiB connection to load the whole directory and cache it.
For compiled languages, binary size would be a problem so it's probably smart to write the compilation output somewhere outside sshfs-mounted directory and run it locally. For interpreted languages generally it shouldn't be a problem since everything is in memory. Python does write stuff in __pycache__/, but in my experience it's rarely more than a few hundred kilobytes per cache directory; it's also possible to turn off writing __pycache__/ with PYTHONDONTWRITEBYTECODE=1.
If it’s just small things like a quick config change or something, I’d personally just ssh in and modify with vim.
For testing, you probably shouldn’t be doing that on a production system, perhaps you could stick the code in a docker container and run the tests locally? If you’re familiar with docker, that shouldn’t take more than a couple hours to set up, and likely much less time than that if the project isn’t super complex.
Feel free to explain your use case further and I bet people could give more targeted advice!
A few questions that would help me give more actionable advice: What is the "tester" you're using? Is it essential to interact with it via a GUI or does it have a command line interface you would prefer to use? Is the OS Linux or Windows? What are the the three layers of RDC?
Feel free to email me at my hn username @gmail.com to discuss further since this is getting pretty specific to your setup! Sounds like a fun problem to try and solve :)
It mounts a remote directory onto a local directory and lets you access remote files as if they're local, with slight latency. Then you can use whatever tools you use locally to edit/compile/run the software.
JetBrains never works on sshfs as it parses files for dependencies.
$ time grep -r foo test
...
real 7m36,154s
user 0m0,110s
sys 0m0,249s
I don't understand why is this happening. Is this an inherent limitation of remote filesystems or is sshfs implementation just sucky?
EDIT: parallelization seems to help - running 100 concurrent reads at a time uses up to ~1MiB bandwidth. I suppose ordinary unix tools don't play well with remote filesystems because they do stuff sequentially.
That being said, if JetBrains is struggling with SSHFS, then they really need to fix their file browsing code.
All the tools are basically designed to work on local files and they tend to work bad interacting directly on mounted file systems.
Editing code directly on a remote server is something I used to do in the 90s before git and other tools made good practices much easier.
My workflow was as follows
- Remote into server and develop from there
- Git to different location, could even be same server
- Occasionally pull on local machine for redundant backup
This allowed me to
- Segregate builds for my testing and the business's UAT
- Allowed me to always have a stable build locally that I could look at from home
Before, I did develop on my local machine and push to a server. Some part of me still thinks that's the proper way to do it.
But I am always seeing what works best for the current situation and developing directly on my server served me well once upon a time.
The next 10 years of “best practices” will return to dumb clients and dynamically provisioned dev environments. It’s simply so much easier.
I'm not sure if the price lines up well enough for a daily-drive for a company, but if I were managing dev environments for an org these days I'd probably jump to something similar. Perhaps something like a decent size EC2 on AWS where devs can tear it down and spin it up again to start from scratch within a couple minutes.
Throw away dev environments feels like a game changer having used it for the past 6 months on-the-go.
It baffles me why anyone would choose an incompatible architecture only to run dumb clients on it. You might as well just save your money and run the dumb clients on 20 year old hardware.
Or you could spend less money on a laptop that is actually compatible with whatever platform you are deploying to and enjoy low-latency local development.
As a bonus, I also setup openvnc, cloudflared (DNS over HTTPS), and pihole. And tmux and mosh of course.
With this setup I can even do my work from an iPad (once you get the VPN setup correctly).
It's nice to be able to reboot my laptop without losing my place in my work.
That it is a question always makes me wonder what was in it for the purveyor's of OS software to make this so impossible to do.
1. VNC has a "stable session" that doesn't collapse when SSH closes. Even between Linux machines, I chose VNC when needing a permanent remote GUI.
2. Remote machines don't always have your preferred applications or the associated preferred configuration. Copying a light .vimrc was always easier than trying to export-import a heavier app.
3. Performance always sucked. Even between machines sitting next to each other.
4. I didn't edit files with a GUI app anyway. Didn't everyone just use Vim, Emacs, or Nano? Was remote GUI apps ever anyone's preferred method (X11 or not)?
3.) No, performance has been fine using X11 apps that were built with X11 in mind (vs Wayland or some sort of locally accelerated GPU helper(do you really need 25% transparent dialog boxes?))
2.) There is remote machine I own/control (it has all the things I need) and going into a system that someone else has asked me to develop on (then ANSI terminal tools over SSL are always working)
1.) VNC has always been hit or miss for me, I do typically use screen(1) to keep sessions alive if for some reason the network dies in the middle.
But all fine points.
I haven't tried it myself, but they seemed to like it.
If you want a full IDE, or just your set of extensions & tools, you could install it there and use X11 forwarding like so:
ssh -X me@box /usr/bin/code
Every now and then clipboard would stop working or resized windows would retain their original dimensions etc. Not entirely sure if it had to do with my setup or network.
I switched to tmux+neovim a year ago and it has been a much better experience.
- SSH remote editing (e.g. ssh + remote Emacs/Vim)
- VS Code + Remote SSH
- Emacs TRAMP
- JetBrains Gateway
- mutagen? (I haven't played with it, I'm not sure if it's suitable for your case)
(I'm voting you up, as I can't vote myself up)
If you are purely doing web design work, I could maybe recommend Nova for its nice CSS/HTML tooling, but again you can recreate much of this with VSCode anyway. When you factor in Nova is 99 dollars this conversation becomes a lot more difficult when VSCode is such a strong free competitor.
Nova has been through several iterations/product resets, I owned it back when it was called "Coda" too. VSCode by comparison has been relatively stable and its ecosystem a lot healthier.
I'd actually recommend VSCode + another Panic Software app for remote file access on a Mac: Transmit. Transmit is mac native standalone remote filesystem access tool, its basically the tool they built into Nova as a standalone, and 45 bucks:
VSCode + Transmit is IMO the best in class tools on MacOS for this kind of remote access work, although you can easily substitute Transmit for a free tool like CyberDuck or similar too and keep software cost at zero.
In the same vain, Gedit and its forks can do this to if that's your cup of tea.
or vim, emacs or nano through a traditional SSH session :-)
Also I'd probably get pissed for using vim over ssh session to program when sometimes the network could sporadically slows down my cursor movement.
Rachel from rachelbythebay.com notoriously uses nano FWIW [1]
If you don't expect a very powerful editor, nano can be a good companion, but it is actually more powerful than most people think it is.
> Also I'd probably get pissed for using vim over ssh session to program when sometimes the network could sporadically slows down my cursor movement.
Also think of using screen or tmux or one of their alternatives in case your session breaks. It's bad enough being slowed down by a flaky connection, losing the session is very disruptive even if they produce backup files. There's also Emacs which comes with a server, I would expect it to survive connection losses but don't quote me on that.
I'm using mostly Visual Code + Remote SSH (which would be wonderful if visual code was a gread IDE, which it's not. A fine IDE, but not a gread IDE). For large java project I switch to Jetbrains Gateway, but the experience is too painful.
Another useful option, depends on your specifics, is the Plan 9 "cpu" command[0], available for Linux at [1]. It takes an unusual approach, which is IMHO superior for embedded machines, but which may work well for some other use cases (though not all).
Now due to changes in where I host my stuff, I can't do that anymore, and I'm struggling to find a simple solution. I have recently discovered that WinSCP can 'edit' a file by pulling it locally into a temp folder, opening your IDE of choice, and then auto resyncing the file to remote server whenever the local copy changes. It's OK as a workflow, but not as flexible as having an open folder to just drop stuff into.
I know the latest version of Dreamweaver has the same sort of thing built in which would be better... But as a hobbyist, I just can't afford that.
This is just so simple and performing, not sure why anyone is struggling with other worse solutions. The only better method would be vscode remote editing plugin.
I only use one-way sync from local to remote though 2-way is also supported. I simply git fetch/checkout on remote first, then git fetch/checkout the same branch on local. You can set up a local script to do both together.
Once in that state, any locally changed files get sent to remote faster than I can switch focus to the terminal and enter a command/enter.
There's also a Remote Development (beta) feature that essentially runs the JetBrains IDE headless on the remote and a local 'display server'. Performance largely depends on the performance of remote and network bandwidth much more than the Deployment Configuration rsync method.
If it's casual edit you need to do ssh is enough, via TRAMP (who can work on various protocols transparently, not just ssh) if you are in Emacs or perhaps with sshfs for an editor-neutral solution (knowing it does not offer a full POSIX fs).
For testing again ssh is a very common and friendly option.
I ended up creating a "runtime" script that first rsync all the files over to my remote machine, then run whatever arguments, `runtime <host> 'make'`. there are some flags too like `-s` to have it not rsync, etc. This works very well. But then this is maybe cheating since my code is still local. The reason for this is my M1 mac can't build our gigantic C++ code base for a whole host of various insane reasons.
You want to be able to iterate quickly and not have to manually run a command every time you want to sync changes. It's also a huge help to tie this into a way to run the unit tests for files which have been changed so you can get feedback and quickly as possible.
Ideally, everything should be made so automatic that you barely even notice the code isn't running on your machine.
Basically just hosting an SMB server and then mounting it as a volume on my work computer. So it looks like a normal folder/directory but it's over a network.
You can also get fancy and configure your NAT to only allow this over local connections, then spin up a VPN. In this way I'm able to securely work in my server from anywhere.
Hope this helps.
Vim, emacs or nano are no alternatives for me as I need to browse large C/C++ legacy codebases and some newer JS. And I rely much on CTRL+mouseclick.
Does anyone know a VS Code extension to use it in a shell instead of the desktop GUI?
VS Code was the only editor or IDE I found for Linux that can be used to browse like that without much configuration. CodeLite worked sometimes but not reliably.
I can just type "code ." in my src folder and can start browsing it. Most IDEs need to setup some kind of indexing. VS Code worked out of the box.
It doesn't support everything VS code can do (notably it doesn't have LSP support), but it could be more accessible to you than vim or emacs.
My vim setup looks like atom in terms of interface: https://github.com/Aperocky/unix-setup/blob/master/.vimrc
Very easy to setup, just add the .vimrc file and run the git clone commands after setting up pathogen (package interface). This setup is pretty nice to edit ts/js/py/rb and C family of languages (add plugin specific to language when you need it). This isn't just for SSH, this is my goto in local as well, but it behaves exactly the same either remote or local.
This doesn't really work for any of the JVM languages however, those you probably want an IDE.. I tried my best in vim and it just don't work.
However, if you absolutely must, use X2go.
When the updates are occasional, fine, but when you call make or run something which updates fast/creates a lot of output, things get hairy, quick. Even on local networks with a couple of connections. Been there, tried that, experienced it all.
X2Go transmits the image, compresses it, and can work with much slower and higher latency network connections. It can also resume sessions.
Tried their new editor Nova and was utterly disappointed. The whole thing is practically unusable. Or I am too old :)
If it's impossible to clone the environment for some reason, VS Code and JetBrains IDEs have remote support. Even vim can edit remote files.
It's not clear from your question what exactly is your problem, would you be willing to elaborate?
For one-off stuff, raspbery pis, etc I tend to use sublime text and remoteSubl
Also found vscode + remote SSH very useful, as others users recommended.
You can use intellj products
Sublime ssh plugins, vs plugins
On android you can use AWD - editor
bonus: collaborative coding by attaching to the same byobu session as your colleague
So sorry no answer, but also interested.
I remember there were a few online code editors, but trust is an issue and they cost extra.
The way you intend to use it though... I have done it in my PHP days, long ago. It's not good practice. Version control systems are a good invention, even if you sometimes just need a quick fix. Edit locally, push to repo, auto build or manually.
There is also ShellFish, which is an iOS SSH file provider. You can use it to access any files via SSH as if they were from iCloud or OneDrive, etc.
Lastly, there is code-server, which is VS Code in a browser. I've used this from time to time, and it works well.
1 machine is Macbook air at home, another is windows in the office