I close SSH port 22 (and what I use instead)
michelebologna.net
michelebologna.net
Anything added in front of your normal service also complicates access, since it’s non-standard. If you want a standard solution to solve all your needs for secure access of IP-based services, use IPsec and be done with it once and for all.
1. <https://en.wikipedia.org/w/index.php?title=Kerckhoffs%27s_pr...>
(Adapted from this old post: <https://news.ycombinator.com/item?id=39898061>)
And for logging in particular: Just switch the logging over to a temp file that lives in RAM (what disk thrash/write-amplification/SSD wear?), or even disable it altogether for failed login attempts.
We already know that there are great hordes of zombies outside of the castle, banging on the doors and the blocked-off spaces where the windows once were, picking away tirelessly. That's been a constant for many years. Documenting their continued persistence is pretty meaningless. None of it is actionable, or stoppable. It's just going to keep happening. Recording attempts from valid users is also largely without merit; it also just looks like noise, and we've got other ways to troubleshoot stuff that breaks without maintaining a long list of zombie attacks to peruse.
If a zombie actually manages to get in, then that's pretty important to keep track of; log that. But the attempts don't mean anything and have not meant anything for a very long time.
(When the word comes forth that the zombies are gone and the noise has ceased, it will be broadcast so far and wide that even the most noise-deafened sysadmins will find it impossible to ignore. In that seemingly-impossible unlikelihood, we can then resume recording attempts to log in with ssh.)
If you want remote logins with encryption and don’t trust OpenSSH, just use telnet and restrict its access to only IPsec-encrypted packets.
Appeal to authority?
In the real world, it doesn't matter. Anything that makes the attacker's life harder is fair game. Stupid dogmatic sheep-like mindlessness only leads to "herd exploitability".
also complicates access
That's the whole point.
No, security mechanisms also should be convenient to use. Otherwise, people will avoid them and work around them using more insecure methods. Anything that can be configured to be transparent, like IPsec, is therefore inherently better than something like this (fwknop), where you have to run special commands from your .ssh/config, and which also only works for SSH.
IPsec is way more complicated to set up than this (or other VPN solutions like Wireguard or OpenVPN for that matter), and doesn't even completely solve that problem, because your ipsec port is open. Although, admittedly, there are probably less bots looking for ipsec than ssh.
Like for any other packets, you can choose to log such failures, or not. In the parent article the logs are simplified not by the decision of using special authentication packets for opening the SSH port, but by the decision to not log these packets, which is a completely independent decision.
What a remote system could try to determine is not whether IPsec is open, but whether it is possible to initiate a key exchange for IPsec keys. You can easily run IPsec without using a key exchange protocol, by using pre-shared keys. I am not sure whether the last official version of IKE (Internet Key Exchange protocol) provides any reply packet to an initial connection packet that was not authenticated, but in any case it would be easy to customize the key exchange protocol, to not provide any reply.
While TLS is the necessary solution for the communication between computers that have different owners, for the communication between computers that are owned by the same entity IPsec would have been the correct solution (e.g. for connecting your laptop with your home server when you are away), except that the original variant of IPsec had too many completely unnecessary complications, which deterred potential users. Later it was simplified, but it had already acquired a bad reputation of being too difficult to use.
Today the main problem with IPsec remains that there are many stupid firewalls that block without good reasons most protocols and ports. So one may have to run IPsec over some UDP port, instead of running it as an alternative protocol. If even UDP is blocked, one may have to run IPsec over some TCP port, or just over the http or https port (just masquerading it as TCP, not actually running it over TCP, which would mess with flow control).
1. Disable all logging about break-in attempts.
2. Do not have any common user names like "root".
Say you want to be able to log in as root from anywhere, just with a password. This is a wise idea; what if you need access, but are in a situation where you are not able to use a certificate?
Make up an alternative name like roto-rooter or whatever pops into your head. Install it into the password file as an alternative name for UID 0. (Make sure it appears later than the "root" entry!). Also edit the shadow file, making sure that the entry is duplicated for the alternative name.
Then in the sshd config file, use AllowUsers to allow only a whitelisted set of users. Here if you say "AllowUsers rotorooter", then the only user id that can authenticate is exactly that one, and it's mapped to UID 0 via passwd/shadow. Add any other accounts you would like to remotely access, giving them similar aliases if they happen to land into a commonly probed space, or are something that a targeted attacker could infer from knowing something about you.
Attackers do not probe the user ID space at all. They concentrate exclusively on probing the password spaces of common user IDs like root, admin, database, www-data, etc. If your system does not support any of those IDs, they are not on a trajectory to crack anything. Your rotorooter password could be "g0d" and they will not get in, if all they ever try is root.
A much better idea is to set up a non-root user and configure sudo correctly.
* https://www.stigviewer.com/stigs/red_hat_enterprise_linux_9/...
There isn't more than one UID 0. Only more than one password/shadow database entry pointing to it.
(It might not be necessary; perhaps there is a way for OpenSSH to remap names, so that our example rotorooter is mapped to root by sshd itself.)
> A much better idea is to set up a non-root user and configure sudo correctly.
Even if so, the same principle applies: do not call that user admin, for instance. Don't use your first name or anything that a targeted, non-random attacker could guess about you.
If that user's name is, oh, 7yMfAxB6, it will never be probed. Though no need to be that paranoid.
From the document:
> Multiple accounts with a UID of "0" afford more opportunity for potential intruders to guess a password for a privileged account.
Whoever wrote that does not know WTF they are talking about. Two identical entries in passwd/shadow do not comprise different "accounts".
The UID is the account; the password DB is more or less just window dressing.
If the two shadow entries have exactly the same password hash, then no, there aren't more opportunities to guess a password.
The only problem with the scheme in a multi-user institutional context is that when an additional UID 0 entry appears that is not "root", it looks like a backdoor someone planted to give themselves continued root access.
I don't think it's applicable to what I'm talking about because an institution probably shouldn't be setting up password-based SSH access to a renamed root account over the public internet. This is something that's a good solution for individuals or very small operators.
> Change the UID of any account on the system, other than root, that has a UID of "0".
Change it to what, with what desired effect? Might as well say 'oh, screw with the password file randomly for shits and giggles'.
Better idea: investigate why there is another 0. Maybe there is a good reason.
... or they know something you don't.
It has the same optics as an unauthorized entry someone planted: a backdoor to retain root access. It will continuously have to be explained to new people who spot it.
If the intent is to keep the passwords identical (which it probably should be), the tooling doesn't support it. When someone changes the password for root using standard tools, the one for rotorooter doesn't sync. This is a problem if someone is changing the password in order to restrict access to just a specific set of people who know the new password. The unaltered entry turns into a de facto backdoor for everyone knowing the old password.
Pitafall: if you put an alias entry in the wrong spot in in the password file, so that it appears before the canonical entry, then UID 0 maps backward to the alias name (e.g. via the getpwuid() function). This breaks all logic that looks for the string "root" rather than UID 0. E.g. shell scripts looking for root in the output of some command.
The kind of organization where it would make sense to forbid tricks like multiple password entries pointing to the same user (such as UID 0) is going ot be the kind of organization where the whole thing is moot anyway in connection with SSH, because in those kinds of organizations, you don't want ad hoc machines to be accessible via SSH publicly. You don't want employees to be solving the problem of SSH ports being probed: do we use fail2ban, port knocking, kazinator's user name tricks posted on HN? ... just no! You have some kind of perimeter VPN. Authorized users connected to the VPN can then use SSH to machines inside the secured zone.
It makes no sense to bring up corporate rules against my solution which for a problem that corporations should not have in the first place: SSH-accessible machines on the open internet being probed.
I don’t recall ever seeing a security requirement not to have 2 root accounts. What you can’t have is multiple users sharing the same account. This is different.
I'm wildly guessing that the BSD flavors that have toor have patched their password DB manipulation utilities and APIs such that when you change the root password, the toor one changes with it and vice versa.
> The reason it exists is shell flexibility. Traditionally root's shell is kept as a statically-linked shell like /bin/csh or /bin/sh so that the superuser can always log in even in single-user mode or if dynamically-linked shells in /usr/local break. The toor account lets an admin have a UID 0 login with a fancier daily-driver shell (bash, zsh, etc.) without touching root's safe configuration.
Could I trouble you to specify? Your link only seems to mention password problems that are trivially avoidable (really, if doubling the guesses is a problem, you're already done).
The reality is that there are basically two ways to operate SSH:
(1) You can, because OpenSSH is the significant remote service with the literal best track record of any remote service, just disable passwords and let SSH run in 22/tcp exposed to the Internet. Probably stop logging people scanning you; there's nothing you're going to do about it, so it's not real information.
(2) You can keep SSH behind WireGuard, an even simpler security protocol with an even better security story (though: OpenSSH is quite solid), which is designed to not to chat with counterparties that don't have keys, even to do negotiation.
Everything else is performative.
I'd incline towards option (2).
There are bound to be many more discovered as LLMs capable of doing so proliferate among those who don't report such things responsibly.
Keeping port 22 open puts you first in line for such exploits, while keeping it behind another layer (whether it's WireGuard or firewall tricks) would buy time, if not keep attackers away entirely. That seems useful, no?
SSH RCE is a super expensive exploit - no one is firing that one while it still is a 0 day without having really important reason or juicy target.
Why, fail2ban here still serve a very useful function: it bans the offending IP from talking to the machine. It's a simple and a very effective heuristic to block both non-offending port-scans and offending too.
> basically two ways
It's always amusing what people like you almost demand what ssh should be run on the port 22 but are fine with a random port for WireGuard.
And for all of you to assume what both 22/tcp and WireGuard are always available and never blocked.
SSH with key-key only or WireGuard/VPN on top are exactly what everyone uses for daily driving.
The broadcasting of the versioning information does constitute a potential leakage of information that could be useful to an attacker. Even if we contrast this to something like mTLS, the client certificate doesn't come until fairly late in the handshake, so there is still information that can be cleaned from the ServerHello from an unauthenticated inspector. This is also the case with QUIC since it piggybacks off the general TLS handshake.
I think the issue is, for a known set of systems, can you create communications between them that are oblivious/non-discoverable to non-authorized systems. I think the answer is yes, but it requires an out-of-band key agreement protocol. Wireguard is an example. However, the problem of out-of-band key agreement can't really be ignored.
I think the article's method is somewhat valid. I also think it is non-ideal for only doing source IP based rulesets, especially in the world of IPv4 and NAT being prevalent.
The goal should be to have your complex layers sitting in front of simpler, easier-to-review lines of defence. Once you get to something like an open connection to ssh, the potential attack surface would be orders of magnitude larger, even though it’s more mature and closely scrutinised.
Password login is disabled. The only way to log in as root directly is via the terminal emulator on the coloc's admin page.
In reality you should use that and then use host based firewall anyway.
In the first iteration the IPv6 got polled by a handful of attackers as soon as the letsencrypt certificate was published. In the second iteration I just picked another IPv6 address from the /64 and made ssh.example.com to point to it. This should work until the attackers start guessing subdomain s...
But yeah scanning IPv6 address space directly without DNS dictionary in hand is tough.
I’m reading this thread and wondering if I’m missing something, why people are still talking about port knocking, port obfuscation, and fail2ban.
I use a cloud VPS. I ssh in via Tailscale. The cloud provider firewall blocks all incoming connections except traffic originating from Cloudflare IP ranges on port 443. My host plays dead to portscans. I check with nmap periodically. I have a break-glass backup terminal login option via my cloud provider dashboard (secured with MFA) in case Tailscale failed and needed investigation and repair.
This has worked well for me on AWS and Oracle Cloud. It’s quick and easy to set up. You need a timed service to refresh the ingress IP ranges, but any LLM could spit that out in a second.
But I take your point… I try to make sure I print out recovery codes when offered, and keep them hidden in a safe place at home.
Cool! Are you able to provide more information on this? My break glass is to temporarily allow public access to my IP, but it's manual and takes time for the firewall rules to apply. I'd love to have a better solution if possible.
Realistically, fwknop is more likely to have a vuln than OpenSHH. Last release was two years ago and the readme dates back twelve :/ Time will tell.
seed=`date +%s%N`; ( echo "secretknock-20260811|$seed"|sha512sum ; echo $seed ) | xargs nc -u 192.168.1.1
On other hand if threat vector includes network monitor with ability to replay i would use wireguard to wrap ssh traffic.
Nothing is good enough on its own.
Geoblocking, fail2ban, port obscurity, SSH keys, limiting logins to specific usernames, not using your public internet nickname, putting things behind CloudFlare tunnels or WireGuard, wildcard DNS obscurity, 2FA... There are many options.
Defense in depth is the only way to put services on the internet.
I use both on one port 22 host. Not much actually touches the server: maybe 5 hosts a day get banned. Meanwhile, China and the Netherlands are forever getting blocked and logged by geoblocking.
I check the 24h log window by country most business days. Some days China leads, other days the Netherlands pulls ahead. Almost never are any other countries close to those two.
1. Disable password auth, only public key auth should be enabled
2. Block public access to SSH entirely, use a VPN instead (Tailscale & co. make this trivial)
And 2 is entirely optional for most people reading SSH guides who just want a server to host their hobby project. Let's be real, you're probably not reading the auth logs anyway so they don't need to be clean, and if someone discovered an OpenSSH public key auth bypass vulnerability, they absolutely wouldn't waste it on you. Just let those dumb scanners go at it all day, they're not getting in.
And a little tangent: fail2ban is 100% placebo and does nothing except clean up the logs a bit. I don't understand why it's still a common recommendation for beginners, it's a relic from the past when bruteforcing was still a concern because people used password auth.
I assume it's suggested precisely because people use passwords. Which, granted, they shouldn't.
Port 22 is getting perma-DDOSed on public IPv4 addresses so use it at your own peril but at least for now most bots don't bother port scanning everyone.
CriminialIP, PaloAlto, Shodan, Stretchoid, Techties, Censy, Onyphe, VisionHeight, Internet-measurement, Infrawatch, BinaryEdge, and a bunch of Alibaba Cloud/Digital Ocean/VPN/Tor exit IPs all know which port your SSH server is on.
1. "would you know if you got breached?" 2. "would you have any reaction time?"
A simple solution that answers this: https://github.com/64mb/ssh-login-alert-telegram/blob/master...
This was before WireGuard and Tailscale, so the main option for remote access was IPsec or OpenVPN, which are both more complicated than most people want to deal with.
Unless it's an April Fool joke, in June.
UDP was designed as no-guarantee of delivery. Why would you rely on it for this important feature?
* Disable SSH root login
* Disable password login and use only certificates
* Enable fail2ban
This have been worked for more than ten years and never been hacked.
I never understand the need for port knocking.
Restrict it to the networks where authorized users will be connecting.
So, I have this nft script which works alongside Firewalld:
$ systemctl enable --now nftables
$ cat /etc/nftables/portknock.nft
table ip portknock {}
delete table ip portknock
table ip portknock {
set knocked {
type ipv4_addr
flags timeout
timeout 6s
gc-interval 2s
}
# Before conntrack: record the knock, then drop the packet.
chain prerouting_knock {
type filter hook prerouting priority raw; policy accept;
tcp dport 12334 fib daddr type local tcp flags syn counter add @knocked { ip saddr } drop
}
# Decision chain for port 41444. Every branch is counted so that
# `nft -a list table ip portknock` shows which path traffic took.
chain gate_41444 {
# Established/related sessions pass unconditionally.
ct state established,related accept
# Host-local. Rarely matches: host-originated traffic is DNATed in
# the output hook before it reaches prerouting. Kept as a safeguard.
iifname "lo" counter accept
# Podman containers reaching the published port (hairpin).
ip saddr 10.88.0.0/16 counter accept
# Trusted subnets.
ip saddr { 10.0.0.0/24, 10.1.0.0/24 } counter accept
# Knocked within the last 6 seconds.
ip saddr @knocked counter accept
# Default deny. If THIS counter is 0 and the accept counters are
# also 0, the chain is not being reached at all -- investigate.
# Do not assume the gate is working just because nothing got in.
counter drop
}
chain prerouting_gate {
type filter hook prerouting priority mangle; policy accept;
tcp dport 41444 fib daddr type local jump gate_41444
}
}
$ ssh -p 12334 -o ConnectTimeout=1 "${sServer}" &> /dev/null
$ sleep .5
$ ssh -o 'ExitOnForwardFailure=yes' -o 'StrictHostKeyChecking=no' -o 'LogLevel=ERROR' -fp 41444 -R "${iPort}:localhost:22" -T "${sServer}" "sleep 14d"
Yeah, it doesn't have all the "cryptographic signatures" of the article; at the same time, it doesn't have some "random" 3rd-party application that faces the internet and directly controls firewall rules that way.
It's still an OpenSSH server with key-auth only. I'm not worried about someone carefully watching my traffic and finding it. I just need Internet bots not connecting to it a million times a second.
> to be unreachable: no banner, no version string,
> It works, but it has a real weakness:
It assumes systemd, which it uses to start sshd. It also restarts a systemd timer to shut sshd down in 5m. Ssh is set up (by defualt on Debian) to have KillMode=process, leaving the client instances alive, just killing the listen server.
I tend to agree with the top post. I've never felt that knock servers really deserve consideration. That said, the way that this knock server is so small, makes use of well known parts of the system as it is (systemd), is so low risk, and built around decent security building blocks (recursively) & not some ad-hoc protocol is kind of interesting and fun. This one is pretty easy to visualize.
This bespoke server should not advertise it in any special way though (generic OpenSSH banner), and can be used to also ban port scanners if they try to do something fishy.
fwknop is impossible to even scan for, though, so it’s a bit more robust in that sense.
Pick a library. https://github.com/Eugeny/russh