In practice I've never felt this was an issue. But I can see how with extremely low bandwidth devices it might be, for instance LoRa over a 40 km link into some embedded device.
Nobody is running TCP on that link, let alone SSH.
Crashing your drone is a learning experience ;)
Remote NSH over Mavlink is interesting, your drone is flying and you are talking to the controller in real time. Just don't type 'reboot'!
and RNode would be a better match.
Life is difficult sometimes.