Autossh – automatically restart SSH sessions and tunnels
github.com
github.com
[Unit]
Description=look ma, no autossh
After=network.target
[Service]
Type=exec
ExecStart=/usr/bin/ssh -o ServerAliveInterval=60 -o ExitOnForwardFailure=yes -Nn -R 7070:localhost:22 pc 'sleep 20m'
Restart=always
RestartSec=20
RuntimeMaxSec=30m
[Install]
WantedBy=default.targetHowever, when you have shitty NAT routers (SonicWall, any AT&T fiber device, for instance), the connections will be timed out or will die and there'll be long periods where you're waiting for the next iteration of the loop, and/or sometimes it'll get stuck and never try again.
autossh deals with this by actually passing traffic and taking action if traffic doesn't move.
The `ServerAliveInterval` option above achieves this.
If there are ServerAliveIntevalMaxCount (defaults to 3) attempts that fail, the ssh connection will drop. And systemd will restart it.
Today you learned. Nice. I've dropped autossh for years and you can too, even on flaky connections.
I've used my own autossh type script for two decades now. It's mostly used to give access to machines behind shitty NAT, and/or that have addresses that constantly change, and/or for systems on CGNAT, like Starlink.
If ServerAliveInterval works so well and negates the need for something like autossh to exist, then why have sessions created by my script, which has ServerAliveInterval (and ServerAliveIntevalMaxCount) gotten hung up where the script needs to kill the old and create a new ssh connection now and then? My script logs each timeout, each session hang, and each new connection, and depending on the network, it can happen often.
Please read the bit where it's explained how autossh sends test data back and forth. Do you think you just magically and cleverly discovered ServerAliveIntevalMaxCount and that the autossh people have no idea that it exists?
Or perhaps they know it exists, they know it's not perfect, and they used another mechanism to make up for the shortcomings of what ssh offers out of the box?
> For example, if you are using a recent version of OpenSSH, you may wish to explore using the ServerAliveInterval and ServerAliveCountMax options to have the SSH client exit if it finds itself no longer connected to the server. In many ways this may be a better solution than the monitoring port.
1. fake data to keep a connection "fresh" for shitty middleware
2. detect connection which are stuck (state = open, but no data can actually round trip) and kill them
3. restart ssh when that happens
Is that what we're talking about here? I think people are saying that points 1 and 2, but not 3, are covered by SSH's ServerAlive* options. And that's also how OpenSSH advertises and documents those options, and apparently even how autossh talks about it in their own readme.
You're saying that those options don't actually solve points 1 and 2, while (your/their/etc) autossh does properly detect it.
Correct so far?
If so that seems like a bug in OpenSSH (or whatever implementation) which should get appropriate attention upstream. Has anyone reported this upstream? Is there a ticket to follow?
PS: I think we're all in agreement that option 3 is out of scope for stock OpenSSH (regardless of what other tools do)
I believe the issue is that the connections often fail or get wedged in other network layers; the only way to be sure that your ssh tunnel isn’t: a) lossy enough to “keep alive” but too lossy to send data, or b) isn’t just always waiting on TCP retry backoff, or c) etc, is to use the tunnel to transmit actual data at the application level.
Isn't that exactly what ServerAliveInterval does? The man page says: "ssh(1) will send a message through the encrypted channel". A plain TCP keepalive wouldn't count as being "through the encrypted channel".
So much smoke & obfuscation. Autossh itself mentions ServerAliveInterval. It's worked flawlessly on all kinds of dodgy connections for me.
If anyone has any damned bug reports, link them.
If I were doing it now, I'd probably use wireguard, probably. This is simpler to set up and works great.
So something like that would solve that
[Unit] Description=look ma, no autossh After=network.target
[Service] Type=exec ExecStart=/usr/bin/ssh -o ServerAliveInterval=60 -o ServerAliveCountMax=3 -o ExitOnForwardFailure=yes -Nn -R 7070:localhost:22 pc 'sleep 20m' Restart=always RestartSec=20 RuntimeMaxSec=30m
[Install] WantedBy=default.target
You also need ClientAliveInterval on the server side (in addition to ServerAliveInterval on the client). In other words, both the client and the server need to be configured to monitor the connection. With this setup I had no issues with reconnections.
systemd's RuntimeMaxSec should help in this case but I've never had trouble with sshd personally
To add more context I use the above service to ssh from my phone to my laptop via my desktop PC. The service runs on my laptop and binds port 22 of my laptop to port 7070 of my PC but wiregaurd would probably work similarly
ssh -O cancel -L 4102:localhost:4000 pc
Host *
ServerAliveInterval 11you want ServerAliveCountMax too but default is 3.
IIRC if there's an active connection on the forwarding thingy, that ssh command won't exit until the forwarded connection is closed, so this won't interrupt an active forwarded connection every 20min.
autossh() {
# Tiny delay after failure in case of connection errors
while ! ssh "$@"; do echo Restarting ssh "$@"...; sleep 1; done
}At this point I configure all of the keep alive and retry options in ssh_config and sshd_config, and use
While true; do ssh user@host ; sleep 10; done
autossh is for keeping unattended ssh tunnels alive, if the connection is flaky or one end is only intermittently available. So for using tunnels for the sort of thing you might otherwise use a VPN for.
Also, spiped is way simpler and more secure than SSH. (On my servers, I tunnel SSH over spiped, to protect the sshd from attacks.)
socat TCP4-LISTEN:1234,fork,bind=127.0.0.1 EXEC:'ssh my.remote.server nc 127.0.0.1 1234'
This will set up a tunnel only when needed, and seems to play nicely with my browser.
In general, when flexibility is possible, such a use-case nowadays would often be better served by deploying WireGuard. Grouchy, out-of-touch corporate net admins probably don't even know what it is and insist on their antiquated Cisco VPNs.
I would still recommend it.
You set up public key authentication with SSH to a reverse proxy, a persistent tunnel, and a socks proxy. In a Firefox profile, you set localhost:port. Done! All your services are available in that browser all the time.
Autossh with a reverse ssh tunnel can also be used to expose an internal service to the Internet through a VPS.
SSH has been very secure over the decades. A good feature of SSH is that it can jump from host to host, unlike VPN.
https://github.com/build-trust/ockam
One binary, easy to use, no ssh getting stuck! (yep, I work at Ockam :)
It will be encrypted through the VPN, so there will be some overhead. But will be as direct as it can be. It only routes through tailscales servers as a last resort, when it can’t find a direct route at all (usually because NAT holepunching fails somehow). Their “DERP” relay servers just relay the encrypted connection. I think you can use your own relay servers, but I don’t know if that feature can be disabled entirely.
Headscale can be entirely self-hosted. It still uses the tailscale client applications- but is compatible.
additionally the acl/auth system, their dns and service discovery thing is nice, though not essential.
I like the idea of it but can't even imagine trying to get it past the cyber security folks.
autossh -f -N -o ServerAliveCountMax=2 -o ServerAliveInterval=5 -o ConnectTimeout=5 -o BatchMode=yes [...]Nowadays I use wireguard an a dedicated SOCKS proxy. The upside is that I can access everything on my home network directly without having to tunnel.
Anyway:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1688774 - [open] "Support HTTP over unix domain sockets" - 4 years old, last activity 7 months ago,
- https://issues.chromium.org/issues/40402523 - [closed; won't fix] "[ENH] Support HTTP over Unix Sockets via http://localhost:[/tmp/socket]/foo convention " - 9 years old, last activity 11 months ago.
I think this is the one remaining advantage of ssh tunnels over using a VPN.
NB if you're doing this sort of thing, you probably want to add `StreamLocalBindUnlink yes` to the ssh options.
ssha () {
while true
do
ssh "$@"
sleep 1
done
true
}
It is much more straightforward than ssh for this purpose, and works well with socket activation under systemd.
I use it with the systemd automounter to encrypt NFSv4, and I have found it to be quite reliable.
So I ran redis in a higher memory box at rackspace separate from my db and my app server. I used autossh to forward 6379 from localhost on the app server(s) to the redis server. Worked like a charm and never caused any issues.
Other commenters are right in that wireguard is a great modern solution to this!
I’ve run autossh for quite some time but it was not reliable enough under my conditions
In good conditions you can go months without sending a single byte of traffic between an SSH server and client and both will pick up the connection just fine when it's time to communicate again.
You could cut off traffic between them for any amount of time and they would be none the wiser as long as the network connection is back to normal when they finally try to send traffic again.
(I had SSH sessions in a QA lab persist as if nothing had happened after the connection between the endpoints was down for almost a week while we replaced the aggregation layer routers. They never saw a link state change since the access layer switches were up the whole time. They never attempted to communicate while the connections between those were down, so there was never any problem as far as they were concerned.)
The keepalives and connection checks and so forth are mostly to account for things like stateful network gear (firewalls, NAT routers, etc) between the endpoints that will cease relaying traffic between them if they are quiet for too long.