Tosh: Changing your SSH server's listen address every 30 seconds based on TOTP
github.com
github.com
Before doing something like this I would worry a lot more about client endpoint security (exactly to what level do you fully trust all the people and workstations/laptops that are authorized to ssh to this thing?), as an overall more likely threat.
There are also lots of less esoteric ways to not have a system listen on any publicly accessible IP address whatsoever. If it's really something critical you should be looking at a combination of making it purely an intranet service only, listening on an IP in an internal network block that isn't accessible from global routing tables at all. Or is completely firewalled off from the world, and only accessible once you've authenticated yourself to your VPN. Or only reachable once you first authenticate (public/private keys, two factor crypto key auth, etc) to a bastion host, and then reach the system from the bastion.
Also, the amount of CPU taken by brute force ssh is not zero. There are plenty of good reasons for this. Even if this by itself isn't the best implementation, it's an example of how to make things much, much harder on attackers.
Bit like having a front door that would withstand a C4 blast, but now all your windows are shattered.
With things other than SSH it can also be effective in the most rudimentary first level filtering out of spam, various things that attempt to relay mail through my server get themselves first banned, and then banned for a longer time after they keep re-trying. Again with primarily the goal of having less cluttered postfix logs.
i've also seen significant reductions in idle cpu by using it and sending offenders to the timeout bin for 24h.
thanks for calling me "most uninformed" though.
What you absolutely never want to do is create an environment that is metaphorically like a uncooked egg, after getting through the outer shell layer, things are soft and squishy inside.
"Defense in depth" is often quoted when people want to add further complexity to a system. There are cases where adding a security mechanism that adds complexity has a benefit that is so large that it's justified (e.g. adding TLS or ASLR). But it always needs to be balanced, because complexity adds attack surface.
The system linked here seems like it's adding a whole lot of complexity and only has a very weak case to be made for what it's good for.
To be clear, we haven't been talking about the OP port-shuffling scheme for many posts now in this subthread. We're talking about not having your servers be externally SSHable, period.
An obvious thing that springs to mind is that the default campus network design that is in all the standard Cisco designs let to basically everything being vulnerable when the last exchange exploit hit or the last solarwinds issue occured. But I would like to have some sources so that I can make a better case to senior management.
I would however also like to hear cases against Zero Trust and Beyondcorp. The most obvious I see with the old approach is that oftentimes Engineers in those environments are not able to work and when security punches holes into the old system the whole thing becomes way more insecure than they're actually aware of.
Maybe, but I find that simply moving off of default port 22 drops the number of people attacking me by at least two orders of magnitude. That's not nothing.
That was the infamous security flaw where SSH keys generated on debian/ubuntu were always out of a set of 32768 keys due to lack of entropy in key generation. So if your SSH setup is compromised like this, the approach in the article would have provided an additional layer of security.
There's a lot of attack surface in there. Port-knocking is supposed to be a way to reduce attack surface. It's a belt-and-suspenders approach to the reality that even fully patched openssh has exploitable bugs.
Using this tool, a MITM with an openssh 0day can just follow you in. KnockKnock [0] and tools like it do not suffer from this defect. This tool is conceptually similar to KnockKnock, using OTP instead of a monotonic counter. Using OTP opens it up to replay attacks.
No, it's a bet that your port knocking tool has less (or better tested) attack surface than OpenSSH.
OpenSSH is pretty thoroughly tested by now, and the pre-auth parts runs with very little privileges.
The specific port knocking tool linked to above seems to expose very little, but there's still some logging going on that wouldn't happen otherwise and the potential for logic bugs in the python stuff. It's not an obvious bet to take.
Does the extra logging carry a risk over and above dos (which is mitigated by the `-m limit` stuff in the iptables rules)?
But where there's an attack surface there is a risk. There's logging and parsing of logs going on here.
Does that translate to practical risk, in the sense that your system will get owned in this way? Personally I wouldn't consider it very likely. A Linux box won't get popped via a plain open openssh but likely not via this python log parser either. It's still not a bet I would take.
There's so much going on in a network stack that I would look for bugs there before the same in pre-auth openssh but one does not know for certain until after the fact.
The fact that there are dozens of similar solutions out there says otherwise. These aren't the kind of tools that people build for fun. They fill a need. Remember, perfect can be the enemy of good.
The oldest machine was Timemaster and 8.5 minutes off. Took me week to figure out why my brand workstation had bad time. Fun times
Could you please share what did you read on this topic?
Thank you!
* - https://www.federalregister.gov/documents/2020/03/23/2020-06...
* - https://barrgroup.com/sites/default/files/case-study-patriot...
Every now and then, certain apps on the a system would crash around the same time. We'd scrape through the logs and usually see our app cratered, some databases, ntpd, sshd naturally. Logging of timestamps was iffy, obviously.
The ntpd was the obvious suspect because, despite keeping a nice low offset to its peers like 1ms or two, for months at a time, out of the blue it would confess something like "time offset is too large, I can't fix that so I'll exit!". After chasing Sun ntpd bug reports[1] for a while, we ruled it out when we saw a pattern in the undamaged logs that looked like
09:59:58.000 ...
09:59:59.123 ...
09:09:01.345 ...
09:09:02.123 ...
After some Sun support and some sunbugs searching [1 again] we found the T4 in that Solaris rev had a hardware RTC with separate registers for H, M, S etc and a write mutex protecting them, but no read mutex. It was possible to read the RTC while it was being updated, which happened when it was syncing the OS clock to the RTC, or something like that. Fixed in a later release.
1: RIP sunbugs database. It was such a mature relationship where Sun would let everyone see what they were working on and customers could participate or at least know about known issues. I would love to find an archive. Of course Oracle shut that off immediately so you had to open a ticket and ask.
Someone on a team I was on spent probably weeks working on a way to process an inbound sensor data stream (which was generated by another bit of software also maintained by us). The data was akin to an odometer, increasing over time based on usage. The trouble was there appeared to be two parallel streams of the same name, which were always offset from each other and while the offset varied they were always roughly close. The algorithm this person came up with eventually sorted them into "thing_1" and "thing_2" streams, at which point it got turned over to me to display. It actually worked pretty well, for what it's worth.
I started asking how am I supposed to show this to a user to make use of, what does this even represent?... but never got an adequate answer. So I started looking at the whole chain, and what I found was the piece of software generating the data had a small bug: it used "hh" instead of "HH" in the timestamp, but also no am/pm. The timestamps were supposed to be 24h and looked like it, but 9:02AM and 9:02PM both came in as "09:02". To confirm, we checked the database and, sure enough, every bit of data was between 1:00am and 12:59pm. In the end we fixed the timestamp bug and threw out the processing code.
It's simultaneously a hilarious date bug, a face-palming colossal waste of time, and a lesson on how not to run technical teams (the manager had no technical skill, and the "team" of ~7 was very siloed and each operated more like teams of 1-2).
Clock is fucked, so TLS certs don't verify due to validity times, so DNS is broken, so NTP cant look up domains, so the clock can't be set...
On the one hand there's not very much else you can do since you're pointing at bits of thin air and saying "there's the trust chain" in the first place, but on the other hand plaintext DNS is... not much better?
Of course that's when the existential "why even DoH in the first place" starts (with side servings of "this feels so wrong putting it on the security report")...
(...Why do I suddenly feel like disabling certificate verification is going to catch on in a big way in embedded ntpds, almost like a standard best practice... aaaaaaaaa)
It's so bad that Google's own authenticator has a "time synch" functionality or something like that in the very TOTP app (and it helps!). This speaks volume as to how bad and how not-solved-at-all the issue of drifting/wrong clocks is.
I much prefer U2F.
The default is 30 seconds, as per the RFC: https://datatracker.ietf.org/doc/html/rfc6238
(not sure that meaningfully changes what you were saying, but just fyi.)
The approach that I've been using for about a decade is a script that gets your current (internet facing) IP and then uses a cloud API credential to add that IP to the cloud provider's firewall as a valid source IP to port 22. The API security and cloud firewall implementation is left to the major cloud providers (and they are very good at these things).
I run it manually whenever I'm in a new location or in the rare cases my home IPs change. You can add the IP to the list or replace the list each time. Or clean it up after a trip with a separate CLI flag. I figured one day I would automate it to track IPs, run constantly, and trigger the change if it detected a new IP - but that day never came because it never annoyed me (ymmv).
This approach could be extended to teams, but there are more details to think through about API permissions, etc.
If nothing else, it serves the useful purpose of stopping the log files from being cluttered up with various botnets' fully automated ssh username/password attempts that are out there, trying to gain access via well known factory default credentials.
As for log clutter, my approach comprehensively stops the logs files from being cluttered with failed attempts; that's one of the main reasons I do it.
Host vm vm-0 vm-0.com
User user
HostName vm-0.com
ProxyCommand sh -c "pyknock-client -s 0.0.0.0 -S \"\$(myip)\" open %h "$(pass my/pyknock/%h)" && sleep 1 && exec nc -4 %h %p"
Port 1792
It’s neat for sure, but not a good defense.
If you could scan 1 million IP addresses a second on a /64 (which is absurd), it would take 600K years to scan a full /64.
> If you could scan 1 million IP addresses a second on a /64 (which is absurd)
Not at all, I regularly scan the internet at well over 20Mpps.
You should never intentionally do it to your own systems. If you see something like this deployed assume it's either malice or laziness.
(Yes, MD5 is safe for this use)
With tcp MD5 your connection is even secured against an active attacker who can sniff. They can't inject, or even RST the connection. Even if they can sniff and spoof everything.
Or in the case of a PNI between two ISPs over their own cross connect, you absolutely want to have a mutual level of trust and cooperation between the BGP peers on both sides of the session.
And then other more modern methods of verifying that the IP blocks you're seeing from some other AS are legit, like verifying their RPKI signatures, IRR entries, etc.
I mean it's the only auth that exists for BGP, so why would you not want it?
You use what actually exists. It's orders of magnitude better than portknocking BS.
Port knocking, fail2ban, nonstandard SSH ports, all of that stuff is theater.
But also, like I said in my blog post (https://blog.habets.se/2019/11/TCP-MD5.html) this isn't just about targeted attacks, but this also hides from things like Shodan, which connects to all your ports and records your headers.
It helps with wide-scale scanning and wide exploitation. Security isn't a yes/no, and getting out these databases without restricting by IP address isn't without actual security value.
E.g. if you do this then next time there's an OpenSSH bug, you won't be in Shodan and other more secret scanning databases to be picked off right away.
Port knocking is just plain overengineered silliness. If plaintext password to unlock another port is what you want, set up a UDP server. "Connecting to random ports" is just fooling yourself about what you're doing.
TCP MD5 least has the benefit that it prevents any kind of shenanigans happening to your connection.
fail2ban, agreed. The value fail2ban adds is keeping your logs more quiet.
nonstandard SSH ports, agreed. Especially after everyone and their dog scans all ports on the whole internet now anyway.
With TCP MD5 you don't even have to consider SYNfloods or SYNcookies. And because only people with the MD5 secret can even connect, it becomes an early tripwire if someone does have the password. Currently if you have an OpenSSH open to the world you should expect your logs to be spamming 24/7, which makes smarter attacks not stick out.
Frankly, given the option I would prefer to not even have port 22 advertise to the world exactly which OpenSSH and OS I use. Not because of security through obscurity, but just to make it slightly harder, and thus harder to get in without tripping any of the tripwires.
Then there's also people who add 6 digit OTP as a second factor. Those are pretty brute-forcable by default, so you can actually do online brute force of a user's password still. Just slower. (OpenSSH has a ratelimit, but I've gotten around TOTP this way). With a system wide good secret this can prevent brute forcing even in the presence of bad user passwords.
But if you've already decided that security is either yes or no, and that OpenSSH is marked "yes secure, and therefore can be open to the world forever, bravely taunting any attacker saying 'this far, no further'", then there's nothing I can say to convince you.
But also not everything on the Internet is (Open)SSH.
Sure, it just begs the question: why in the world would you still try to find nails for the hammer called MD5 when (according to Wikipedia) cryptographers recommended upgrading to SHA-1 in 1996 already? This project's first commit was well, well beyond the deprecation of MD5. It's a bit safer than but also not entirely unlike putting a Windows 7 on the internet just because there are no known exploits in an up-to-date 7 system currently.
Despite SHA1 today and MD5 for even longer being “insecure” for some use cases there are plenty of usecases for which they are still secure and will remain so.
The reason for a wide deprecation is that most people can’t evaluate a hash function or an encryption algorithm in context well so it’s easier to say simply don’t use X.
RSA is also crap for many things and it’s slowly but surely being deprecated but it doesn’t mean it’s completely broken for every usecase.
Just to be clear I’m not stating that MD5 is necessarily safe for the usecase the GP states it is.
If confidentiality isn’t a factor (since any hush function that is fast m enough to brute force isn’t going to be particularly secure) and if integrity cannot be compromised through collisions then the hash function is safe for this usecase.
Why use MD5? It’s relatively easy to implement securely m, there are a lot of safe implementations and it’s fast.
This is why CRC32 is still used today also.
There's no reason to choose MD5 over SHA-1. It's less secure and slower and there's plenty of free implementations of SHA-1.
Ideally you'd use SHA-256, because of (smaller) security concerns with SHA-1, but it is a small touch slower than MD5.
% for i in md5 sha1 sha256 sha512; do echo -n "$i: "; time ${i}sum test.bin > /dev/null ; done
md5: ${i}sum test.bin > /dev/null 1.37s user 0.13s system 99% cpu 1.501 total
sha1: ${i}sum test.bin > /dev/null 1.84s user 0.12s system 99% cpu 1.952 total
sha256: ${i}sum test.bin > /dev/null 4.43s user 0.16s system 99% cpu 4.593 total
sha512: ${i}sum test.bin > /dev/null 2.69s user 0.12s system 99% cpu 2.810 total
- Lots of modern x86 machines with SHA-NI that were typically 50-300% faster than MD5
- Older Intel machines, where SHA1 was generally slightly faster, with one or two exceptions where the reverse was true.
- ARM machines with good SIMD/NEON where the NEON implementation of SHA1 was 30-40% faster than MD5.
- Embedded ARM machines with bad SIMD/NEON where it was pretty much a tie.
- A few ARM machines-- most notably Broadcom chipsets in the Raspberry Pi, where MD5 wins big a large margin.
- Embedded MIPS 24k, where MD5 won by 33%.
Then I found http://bench.cr.yp.to/results-hash.html , which bears out what I'd measured.
In any case, I don't think "speed" is any reason to select MD5, unless maybe if you're on MIPS 24k.
SHA512 is expected to be faster than SHA256 on modern 64 bit architectures due to fewer rounds per byte.
What TCP stacks support TCP-AO? They do support TCP-MD5. That's why.
I thought this should be clear from the fact that it protects against RST packets. Nothing on an application layer can do that.
I wish I could edit that comment because while I expected people to go "oh, I didn't know TCP had that!", multiple commenters seem to have not read past "MD5" and assumed that this is pure application-level.
Kernel needs support. This cannot be done in user space.
TCP MD5 has been used for decades to protect BGP, and exactly because it's still safe there's been no push to add TCP-AO.
And it inherently requires kernel support, because it's part of TCP, not the application.
I would have preferred TCP-AO (RFC5925), not TCP MD5 (RFC2385), but the former is not supported anywhere.
See more at https://blog.habets.se/2019/11/TCP-MD5.html
(I should have added this link in my comment, but I forgot I wrote a blog post on the topic)
I'd almost rather a combined thing where it's HOTP but it also rotates once per day like at midnight? Does anything do that, or does it even make sense? Is there a reasonable alternative -- challenge-response maybe?
If you experience that often, I would probably disable the setting to automatically set time from the network.
> booting between Windows and Linux screwing up the system timezone setting
That’s easily fixable with one registry change (RealTimeIsUniversal). You can also tell Linux to use the local time, but Linux will be less happy about that than Windows (Linux won’t write to the real-time clock automatically, for example).
If for some reason your time is off (e.g. after 3 failed attempts), it's easily detectable and fixable. Just browse to time.is [2], and your time is off, and set it manually if needed.
Because there's an increased dependency on accurate time, bad network time is now quite a rare occurrence in my experience. I haven't seen it happen in the last 3 years.
Once you point NTP to a trustworthy service (e.g. time.google.com [3] or time.cloudflare.com [4]), you won't have any issues.
The Google time server offers leap smear [5], and the Cloudflare one offers NTS (authenticated NTP).
1. https://wiki.archlinux.org/title/System_time#UTC_in_Microsof...
3. https://developers.google.com/time
4. https://developers.cloudflare.com/time-services/nts/usage
If you really want 2FA for SSH, use something like Yubikeys that increment a counter and generate tokens based on that counter. And use it during the actual authentication session, not for figuring out which magic port the server will be listening on. You never have to worry about synchronized clocks, just a database tracking the highest counter value ever seen, so that previous values can't be reused.
Would I employ this for a critical commercial server? No. Would I do so for my private server? Absolutely, without hesitation. If it really loses track of time so badly (hasn't happened yet to my knowledge), I can just log in through virtual console.
[1] I'm excluding drift and inaccuracy, since from my anecdotal experience they are usually not that bad.
[EDIT ADD]>Actually was some Symbian and then Java implementation of the skey. Had Psion Series 5 used initially.
In ... 5-8 years, I haven't had a single hit other than me mistyping a password. I understand the issues around doing this, but I'm actually shocked at how little (zero) incursion attempts it gets.
I should also note that since this is a personal machine, I also have entire countries blocked by CIDRs. I also do this for my wife's company's retail presence, and blocking the big 4 (China, Russia, Iran, NK) at a CIDR level stopped something like 98% of the brute force ssh attempts (the vast, vast majority of those from China).
I know it's not foolproof, and I may get some false positive blocks, but she doesn't have a business that needs to allow people from those countries.
Knocking has multiplicative growth and so many more possibilities.
Perhaps you could include honeypots in the IPv6 range where you’re not bound that block the user, but this seems less reliable overall.
I’m sure this is just great fun, but perhaps not something to think of as secure so much as a fun idea — to make it secure you might want to use a system more like knocking.
Look at Single Packet Authentication. Fwknop is a solid implementation.
gofwd: A cross-platform TCP port forwarder with Duo 2FA and Geo-IP integration. Its use case is to help protect services when using a VPN is not possible. Before a connection is forwarded, the remote IP address is geographically checked against city, region (state), and/or country. Distance (in miles) can also be used. If this condition is satisfied, a Duo 2FA request can then be sent to a mobile device. The connection is only forwarded after Duo has verified the user.
It makes more sense when used broadly by the majority of traffic across a link, by multiple clients, but it could be a useful application for IPv6 in environments an intermediate network is untrustworthy.
No. The mechanism underpinning TOTP should guarantee that the internal state does not leak from any outputs produced. That is, of course, unless some security flaw is found, but that seems unlikely to ever happen at this point if you use a regular SHA-2-based TOTP. It basically does HMAC_SHA256(secret, time) where the time is known (also to the attacker) but the secret is shared between the two authenticating systems. If you could derive secret from time+output, the mechanism would serve a much more limited purpose. Part of the purpose of TOTP is that an attacker that observed a (number of) login(s) can't predict any future or past tokens for their own use.
Not perfect, but good enough to deter people who'd first naively try to ssh to my server and then try to nmap it to find out if I have ssh running on some other port. :)
900 IP addresses blackholed just over a 1 day period. "security" scanning is incessant.
https://en.wikipedia.org/wiki/Frequency-hopping_spread_spect...
Interesting side-note, one of the people who re-invented this technique was actress Hedy Lamarr who was quite an accomplished inventor
> The [ipv6] address space therefore has 2^128 = 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses
That's not even counting the 65535 ports you could then use.
I'm not saying this little demo is a disaster or anything. But for example, perhaps it requires an awareness of this scheme in an external firewall's rules, and maybe another machine pops up in the rather large IPv6 range that's now available.
At its extreme, these sorts of approaches can bring a lack of clarity which layer is providing the actual security.
The number one step any public‐facing SSH server should take is to switch from password auth to keys only. Anyone who’s still concerned can put it behind a WireGuard VPN. Layers typically added beyond that (like changing port, etc.) don’t even register on the security scale, so to speak.
The tweet that inspired the post mentioned port knocking which has always been rather ridiculous given those alternatives.
I wonder if there's a "missed opportunity" for hackers there given presumably how many forget to configure their IPv6 firewall.
Suppose you can write code that can try to connect to the SSH server on one million IP addresses per hour, if they respond you attempt an attack. You can try all of the servers in the entire IPv4 Internet in a few months even with a pretty naive algorithm.
But if you do this with IPv6 you won't ever finish trying addresses and indeed almost certainly won't find even one server (let alone successfully attack one) in your lifetime.
So immediately you need a more expensive attack method. Maybe you buy a supply of "passive DNS" (name -> address answers stripped of information about who asked, many big DNS providers sell this) which is not cheap and not well suited to this problem but it gets you somewhere. You pull out IPv6 addresses and try to SSH connect to them from your supplied list. This could work, but now you need to hope that your potential victims revealed themselves to you, all the juicy SSH servers in the world are invisible otherwise.
If "real security" was a real thing and anybody knew how to actually do that, things like the Colonial Pipeline ransom-ware attack, etc. wouldn't happen all the time. As people have been saying since David Lightman in 1983 "Hey, I don't believe that any system is totally secure."
That's not to say security by obscurity layered on top can't be useful
I think that most people who would implement this (or similar schemes) realize exactly that, and are practicing "defense in depth". Could it be a "dangerous distraction?" Sure, in principle. But I don't see any particular reason that this would be more so than other elements of a "defense in depth" strategy.
Of course it could. Unless you're really going to posit that there are no bugs in any widely deployed ssh server implementations. Doesn't seem very likely to me.
Anyway... if you're being specifically targeted by a highly advanced adversary, it probably doesn't matter what you do. I tend to assume that most of us, most of the time, are not in that position, and should employ a layered, "defense in depth" strategy. Whether or not this specific technique is something worth deploying or not is an open question to me. My position is simply that we shouldn't just dismiss it out of hand without deeper consideration.
You also know they don't mean port numbers because there's no such thing as a 6-digit port number.
From tfa: """Imagine your SSH server only listens on an IPv6 address, and where the last 6 digits are changing every 30 seconds"""
The truth is that playing defense is as much an exercise of technical design as it is economics.
Yes - if someone finds the SSH port, they have a window of opportunity, and you will be owned if you are not properly securing your server through the normal channels.
However, now they only have a small window of opportunity (say, 30 seconds). This does a few things:
- it takes time (money) to attack a target. without access to the OTP secret, randomly assigning ports dramatically increases the cost (time) of attacking you. throw in a tar pit and it's even worse. if you're not a high value target, the attacker moves on.
- now, any failed authentication attempt to your SSH server is a highly credible threat. repeat attempts are even more suspect - you are being targeted, and they probably have the OTP secret. effectively, you are able to resource your team more efficiently, because you can filter out noise.
security is not black and white. if you are a valuable enough target, someone will find a way in. defense in depth helps you manage your defense with limited resources.
Indeed, but might there not better/cheaper ways to secure SSH?
Something that doesn't involve custom configuration that needs to be maintained.. like VPN to a jump-host... Or..?
Configuring and maintaining custom hacks is not cheap.
Zero window seems better than a 30 second window.
Excellent points otherwise.
My primary problem with people going for "security through obscurity" is that very often there are things which are intentionally obfuscated or obscured, in such a way that the implementer thinks that their method of hiding things will provide a significant measure of security. But then all of the other more common sense security precautions that should be implemented before the obscurity are ignored.
If I had a dollar for every industrial/embedded/M2M/IOT type thing that tries to be secure through obscurity but has other gaping holes in it, once you're familiar with the technical workings of the product...
0: assuming I'm not being overly charitable
There must be better ways to leverage long term shared secrets, recent authentication success, etc. I'd like to see something like Signal's ratchet mechanism.
Also, there are these internal network creation systems called VPNs.
Sensitive ports should be guarded behind VPNs on private networks. And, the VPN port itself should be guarded with SPA port knocking.
Stop putting ssh on everything and on public IPs on the actual public internet. Don't do this. Do you know how many weeks were spent cleaning-up after idiots who did this with desktops contracting W32/Blaster? One "secured" Oracle database box on a public IP got an unknown trojan rootkit Mark Russinovich was like: what is this voodoo that they do? The "only" solution, since it "couldn't ever be taken down," was to block everything it didn't explicitly need to function and general outbound internet access. Lack of security, idempotent automation of configuration management, and restoration caused these issues. It languished on for years with what effectively was an "endogenous retrovirus" that "couldn't" be removed.
I prefer to keep the word 'secure' for things that provide at least man-in-the-middle protection, which this approach doesn't.
Leave ambiguous terminology hair-splitting at the door and get to specifics.
It's securing the keyhole of the padlock. No sense moving the padlock around when it can be closed to begin-with, and opened with a special knock that is extremely complicated: time, service, and identifies which user.
Port knocking on top of VPN, SSL, SSH, Wireguard, or whatever. You don't do this with telnet because common sense. Duh!