Regarding the last part of your comment: you may get some value out of this plugin, https://github.com/chipsenkbeil/distant.nvim, https://www.youtube.com/watch?v=wVAsbpByQ3o.
I've not used it but your use case made me wonder if there's a nice way to do it, as I regularly SSH into my NAS and RPi's, so I had a quick search.