A recent abrupt change in Internet SSH brute force attacks against us | Hacker News Reader

A recent abrupt change in Internet SSH brute force attacks against us

utcc.utoronto.ca

53 pointsrolph31 comments

> "The U.S. Justice Department confirmed on Wednesday that it disrupted a botnet run by a prolific Chinese government hacking operation known as Volt Typhoon."

https://therecord.media/china-run-botnet-takedown-fbi-doj-ro...

This would be a great intro scene in a movie about hackers: the ominous drop off in attacks noticed by a random admin due to all bots gathering their strength to deliver a huge tsunami of attacks agains the US power grid.

I don't put much stock in attribution, but keeping an eye on what attacks stop when an adversary ends up refocusing their attention due to scandal and/or war is a pretty reliable indicator in my experience (and very difficult to false-flag).

Activity we previously attributed to the Russians didn't flicker when Russia invaded Ukraine-- but did stop very quickly into Israel's invasion of Gaza.

SSH attacks seem actually to have clearly spiked in the wake of Ukraine's invasion

https://www.bsdly.net/~peter/Predicting_developments_in_real...

Interesting. We don't run public SSH so it wasn't a consideration.

If true, that seems like some solid evidence. Do you have any links/sources I can read up on? I wasn’t able to find anything.

This type of analysis quickly starts to wade into “intelligence” in the CIA sense of the word. It’s possible to do some of this digging on your own, but for the most part this type of information ends up requiring a lot of inference and synthesis across different sources and collection mechanisms to build a coherent story.

If there’s one thing I learned from working in this space, it’s how there’s just massive amounts of “history” playing out every day that will never be written down or acknowledged. In particular, cyber/electronic warfare is a very active space, and quite a few nation states regularly commit what many of us would imagine would be considered acts of war against each other, without a word said to the public.

It's not evidence at all, just data (intelligence) that makes me reconsider what I thought I knew about the situation and what else might be related that I didn't consider before.

We have a "special" relationship with Israel so I can't go too much into detail, but suffice to say it was password spraying attacks that originated from domestic residential IPs that dropped off. Normally foreign agencies use datacenters and a known set of VPN ASNs. Israel happens to have their own onion routing network in the form of Hola/Luminati, but that isn't a discrete ASN-- it's a botnet of residential proxies.

https://news.ycombinator.com/item?id=18161706

I don't know if Luminati is even still a thing but this is the sort of footprint I'd expect from it. They'd find residential proxies useful for their astroturfing campaigns so I assume it's still up. Attribution is a game of educated guesses.

Now, I'm not implying the Israeli government is the actor here. For all I know it's some bored teenager fucking with us. The timing is what's suspect. Either the operator was compelled to stop when war broke out or the infrastructure they were using was somehow impacted by the Gaza offensive.

> Activity we previously attributed to the Russians didn't flicker when Russia invaded Ukraine-- but did stop very quickly into Israel's invasion of Gaza.

What's your conclusion here, though? Who do you believe now was the actual bad actor and why did they stop? You're just leaving it open.

I kinda have to leave it open. I can't rightfully say for certain it's Mossad.

It looks like it's related to Israel's offensive, but even that is subject to interpretation. It remains an open question to me too. The most I'd feel comfortable speculating on is that the attacks were possibly leveraging Israeli infrastructure (Luminati?), which says little about the actors involved.

There's a secondary insider event that occurred coincidental to the beginning of the attacks against us that makes me particularly suspicious of our "friends" but I absolutely cannot discuss that here.

I see steady, slow attempts punctuated by some rapid attempts.

But I'm still amused with their wasted efforts by not checking error messages; my servers only accept keys

Back in August of 2023, I would gets *hundreds* of SSH login attempts per minute. Looking at my logs, the numbers have dropping precipitously ever since.

As of this month, I only received 17 attempts. All from the same IP range.

Can't say I can complain about this.

Moving from 22 to a nonstandard port did miracles to avoid this.

Really think port knocking could help thwart this entire category of attacks too.

I run ssh on an overlay network so it is never directly exposed to the internet and it is nice to not have to worry about changing ports, fail2ban, knocking. Of course you don't get to do fun analytics on the bots though.

Im using a non standard port in the 40,000 range. Changing the port won't necessarily stop the traffic. A lot of these bot nets will portscan an entire ip of anything that has port 80 open to find non standard ports.

It is much more expensive to port scan against UDP than TCP so a nonstandard UDP port would be a lot safer again. OpenVPN is one of the only services that let you do that though.

I'm pretty sure you can put Wireguard on any UDP port you desire, too.

https://www.abuseipdb.com/

https://www.misp-project.org/

https://www.crowdsec.net/

Why are they not using fail2ban?

Here's a good list of reasons: https://www.opencve.io/cve?vendor=fail2ban

Parsing log files not intended to be read by automation (eg. questionable or nonexistent escaping) to then take actions as root is with great remote injection vector is not a good idea.

That's a very very small list of vulnerabilities for such a widely used piece of software.

Using fail2ban with SSH is very widely done and is thoroughly recommended. It increases security and is objectively a good idea.

> That's a very very small list of vulnerabilities for such a widely used piece of software.

They are schoolboy errors in a relatively trivial piece of software that very well demonstrate the fundamental security flaw in its design.

> is thoroughly recommended

Recommended by whom? The Internet, or prominent experts in the field?

> They are schoolboy errors in a relatively trivial piece of software that very well demonstrate the fundamental security flaw in its design.

I read through the CVEs and they're not 'schoolboy errors' at all - in fact, the first one doesn't even refer to fail2ban itself but to a program being called by fail2ban.

There's no fundamental security flaw in its design.

> Recommended by whom? The Internet, or prominent experts in the field?

It's recommended in several linux/unix books - physical books, written by experts - and in many blogs. So, both.

I tend to work with whitelists instead of blacklists: I add only the IP ranges used by my ISP to the firewall whitelist. Most countries can't even ssh to my machines if they wanted to.

What threat model does fail2ban prevent, that couldn’t just be solved by using secure credentials?

Let's say you lock your house's front door with a secure pass phrase while you're still inside. Once per second some stranger walks up to your door and knocks. You now get up from your chair, go to the door and say "Pass phrase please?"

The knocker says "password123" and you say "Wrong, go away." But one second later they come back and knock and you have to get up and go to the door again. And again. And again.

The bad guys never get in because they only try stupid passwords, but the constant getting up and going to the door gets pretty tiresome.

So you get a dog, you put it outside your door and you train it to keep away for one minute any stranger who knocks but fails to get in.

Now you have a lot more peace because you're not getting out of your chair every second to ask the pass phrase from some rando.

Fail2ban is the dog.

I have seen a lot of failed connection attempts in my logs, but never noticed substantial resource usage. Is SSH DoS a serious concern? I get the impression people are suggesting it to protect authentication, which makes basically zero sense to me.

F2B shrinks the attack surface. There's almost no chance of somebody getting in if you secure your server properly, but "almost no" is not zero, and making it even closer to zero seems like a good idea. My main concern is that some future update to SSHd will contain a zero-day vulnerability and F2B would at least slow the bad guys down. My second concern is that I might spin up a server and forget to disable SSH passwords and leave one enabled that happens to show up in a password database. Oof.

Where did they say they did not use fail2ban?

I don't see any mention of it in the post

Must be nice, I haven't noticed that much of a change on my end.

Reply on news.ycombinator.com