Ask HN: How to protect against endless SSH login attempts to my server? | Hacker News Reader

Ask HN: How to protect against endless SSH login attempts to my server?

19 pointsJoeOfTexas27 comments

I changed my SSH port and use fail2ban, but it seems these botnets have endless supply of IP addresses or there are just that many countries scanning.

I think changing the port blocks them for a little bit until they scan all my ports again. They found me at 2222, so I had to change again. They were using up about 8-10GB of bandwidth per month just on SSH login attempts.

Bonus bandwidth usage: https://i.imgur.com/T0sJCpM.png

Seems like 1-2GB per day sometimes, hah.

View on HN

If that's bothering you, you might consider configuring port-knocking[0], or setting up a VPN like tailscale instead of exposing an SSH port.

[0] https://goteleport.com/blog/ssh-port-knocking/

First time I hear about port knocking, that's funny and pretty cool.

I have a firewall that allows only a small range of IPv4s to my SSH servers.

Then I have a bastion server with Teleport to access the servers when I'm not on an allowed network. I could have used Tailscale or OpenVPN or even OpenSSH again, but I went with Teleport. Mostly because it also supports Kubernetes well and does user management and MFA nicely.

I also, for fun, did setup a GeoIP firewall on the bastion. It uses iptables and xtable-addons and a maxmind database. I blocked many countries that are known do such login attempts. I of course blocked myself out of the bastion during the setup of the GeoIP firewall.

Limit the SSH ciphers, MACs and key exchange algorithms you allow. This will drop 99% of random SSH connections since session negotiation will fail immediately.

I would change the ssh port to nonstandard, maybe implement a knocking protocol.

Maybe leave a honeypot/tarpit server on the original ssh port to distract the bots.

8gb of bandwidth a month seems not worth sweating about.

Yea, seems like non-standard port is the way to go for now. Knocking protocol seems overkill. I'm security conscious, but I am lazy too.

I don't disagree. It helps to put the new ssh port on some other well established port you are not using (say port 52, 69 or 70 or something). Bots seem to be less likely to try out ssh when another protocol is supposed to be running there.

Speaking from experience, my ssh attempts have been reduced to pretty much 0 after the switch.

- Start with hardening SSH (Client first, then the server) to ensure you do not lock yourself out by selecting ciphers the daemon does not support. [1] This will actually speed up ssh client-server negotiation. Assuming you have a recent version of OpenSSH this will drop the older bots.

- Ensure you have SSH key trusts and then disable password authentication and don't bother blocking anyone. It's just noise and the C&C nodes will just use a different node to reach you.

- If the noise is distracting, move the sshd listener to a high port. This is not security and will not stop a targeted attack, this is just to stop the noise. 99.999% of the bots will just go away. [2] Ensure the port you select is opened on the firewall first.

- Optionally, add CIDR/IP restrictions to your SSH keys if you are concerned that you might leak your private keys somehow. This is done on the server side in the authorized keys file(s) depending on how you configured key trusts. Maybe leave one low-permission account unrestricted in the event you want to access from a cell phone.

    grep -i authorized /etc/ssh/sshd_config
    AuthorizedKeysFile /etc/ssh/keys/%u

    cat /etc/ssh/keys/bender
    from="172.16.0.0/12,192.168.0.0/16" ssh-rsa AAAA[snip]... comment

- Very optionally, only permit connections to ssh that have valid MSS. This drops massscan, the most prevalent scanner used by bots. If your clients use jumbo-frames then you may need different values such as 1280:9200. Most home connections will be 1460 and some mobile ipv6ipv4 gateways are 1280 or 1380. Adjust as you see fit after using tcpdump to see valid values vs. bot values.

    # see what is valid and invalid by capturing syn packets to 22
    tcpdump -p -i any -NNnnt -c512 port 22 and 'tcp[13] == 2'

    # limit ssh to a desired mss range.
    -A INPUT -m tcp -p tcp --dport 22 --syn -m tcpmss --mss 1280:1460 -j ACCEPT

[1] - https://www.ssh-audit.com/

[2] - My anecdotal experience since the 1990's.

Also another good solution is to use key authentication. Anything with password authentication will drop connection.

https://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server...

I'm amazed at the amount of people not even reading the full post and suggesting fail2ban or a different port (already done) or to setup a honeypot and generate even more bandwidth (worse)!

Just use hosts.allow for select IP addresses while banning all else via hosts.deny. This is a very low level blocking solution that is native to Linux, very simple, and very secure. I believe it also operates "closer to the metal" and under xinetd, but I could be wrong on this.

Port knocking. This is something we did back in the day to avoid opening ports. You can basically send a sequenced set of packets to different ports that will then unlock the firewall so you can ssh in. You can get things like knockd for this.

Run Tailscale on your server, bind the ssh port to the Tailscale local address only, then you can just logon to Tailscale to access SSH, no longer publicly accessible.

Or just use tailscale ssh, and turn off the old sshd.

It’s an uphill battle and one that I personally don’t think it’s worth fighting. If you’re running a service on a public address, there are going to be access attempts whether you like it or not regardless of them being legitimate or not. It’s just the nature of the wider internet. I have just learned to live with it, it will never stop.

You’ll end up moving around the point where probes happen if you have a bastion, if you use tail scale, if you run fail2ban, move the service to another port, implement knocking or any other overused suggestion.

The only real solution is either keeping your sshd up-to-date to minimize the risk of zerodays, moving it to a private address (and by virtue of still needing to get to it, exposing yet another service), filtering everything but a handful of addresses or just not using it.

How much time do you want to sink into this, what’s your threat model, is that used bandwidth actually worth anything are the big questions I’d ask right away to inform an answer.

I would recommend changing the default port of your SSH connection from 22 to anything else and running a tarpit such as endlessh. This is a great video on setting that up:

https://www.youtube.com/watch?v=SKhKNUo6rJU

Why not block all ip's that don't belong to your isp since you're the only one who'd need to ssh to your server? There are some good sources out there to find out your ip range

I was wondering if that was possible. But, I don't think the range of ips would be accurate, since they might have multiple gateways(?). Also, if I'm traveling, I'd need a home VPN to whitelist.

You could try to put them in a tar pit!

https://github.com/skeeto/endlessh

I'm also using this and it's fantastic!

Depending on your use case/business model; geo IP blocking. Blocking non-USA IPs wholesale cuts down on it significantly in my experience

Hmm, probably not a good idea for my website, but blocking certain countries might be best.

Do a reverse whois lookup on the IP. Ban the whole block. Many times, if you block just 1 IP address, they just move to a nearby one. By blocking the whole block, you prevent this. Do this enough times and they move on. Also make the ban permanent.

Do geoblocking. Block all of China, Russia, and North & South Korea. 90% of hacking attempts come from these areas.

use tailscale ssh, you’ll need to connect your clients to the tailnet. But in exchange your ip won’t be publicly visible!

Use UFW with fail2ban and set a whitelist for IPs that can connect to ssh. Stopping connection attempts is like trying to stop a toddler from touching everything. IP and Portscans run continously day in day out. The servers I manage are scanned all day long and as safe as the security you set.

Only time this annoyed me was when the provider started throttle my address. Specifically Pirate Bay did not work anymore.

If you change MAC, the provider will assign new IP-address. And then you use different port every day. Port will be day number plus X, which easy to remember and easy to automate.

I was anxious about it too long time ago, but then asked myself if it’s worth my time and convenience.

Reply on news.ycombinator.com