Hacker News Reader

Another fun one is sending a redirect to a gzip bomb or extremely large file (i.e. speedtest download file) for when certain non-existant URLs are requested, i.e. /wp-login.php on a non-wordpress site.

Earlier this month I was downloading top 1 million websites' robots.txt and some guy just served 100mb of whitespace :/

I love it!

Once I was trying to scrape from some internal API, and made a request that used parameters the real client wouldn't make. I didn't expect curl to scream "FUCK YOU <some name>" at me. Presumably the name belonged to someone who scraped it before.

LOL, that is great.

Is it legal to host a gzip bomb? Seems like a malicious file to serve up…

It can cause your site to end up on Google's safe Browsing black list which can be a death sentence for a business. Google has automated process for identifying malware and black list such websites. Almost all browsers use this list to warn users. This is why it is also dangerous to host anonymous uploaded files even for a short time. https://news.ycombinator.com/item?id=25802366

> It can cause your site to end up on Google's safe Browsing black list

If it's a file that nothing on your site links to, and doesn't really "exist" how would google ever index it? Especially if you put in as a deny in robots.txt, which as far as I'm aware, Google honors.

MySQL has a weird feature, by default enabled in many clients, that the server can request a file from the client.

I have requested /etc/shadow, cracked the hash for root password, and ssh'd back into the botnet node that was bruteforcing passwords. I then shared the information with the webhoster where the botnet was running and a local infamous antivirus company (Avast before it was leaked that they are evil) and got a t-shirt.

https://www.abclinuxu.cz/blog/jenda/2019/2/exploiting-mysql-...

Outstanding job! My opinion of mysql continues to improve :-/

As a curiosity, I found the submission link in the comments on that thread https://news.ycombinator.com/item?id=19305823

Nice. Also an illustration that its good to establish one's inner motive first, like here might range from a desire for simple solutions, to another dimension of drama, which might indeed be desirable sometimes, for entertainment or for snapshots to use in a book one is writing. :)

Try negotiating ssh compression and then sending terabytes of compressed zeroes. Might work for a badly implemented scanner.

Semi-related, I've just started serving large files to webserver scanners, some are so poorly made that they just download the entire file

Please tell me it's a massively upscaled hello.jpg you send...

You still got the response packets?

Disable password authentication and fail2ban becomes completely unnecessary.

You still get the attacks and they show in the logs

Also, it’s not just ssh, there are brute force attacks for pretty much any service that you run

Our logs show automated attempts to run exploits on our web servers everyday

This. I really do not understand why people use fail2ban when the threat is somewhere else.

It won't stop a ddos but will certainly, at some point, prevent you from logging in.

It keeps the logs cleaner. If you either don't look at the logs at all, or have fancy log analysis systems, then it does not matter. But if you are in the middle, and just manually look at the logs every once in a while, this would be a great help.

If you are logging in via ssh, the chances of being locked out arr low - using password auth is a bad idea, and once you set up ssh keys, the connection will always succeed. And in case of rare event like new system setup, you can always ssh via some other system, -J is great for that.

> the connection will always succeed.

Not from my experience. If you have too many keys and certain ssh agents like gnome keyring don't pick up the key intelligently and will try a all the keys in some order often resulting in the server giving rejecting you due to too many failures.

Yeah, this is annoying.

But you don't live with it -- you either move the extra keys to subdir, so that gnome keyring does not pick it; or use "IdentitiesOnly yes"/"IdentityFile foo" in .ssh/config to restrict certain hosts to certain keys (and yes, those work with ssh agent caching too).

I know many people just don't care about working tool, and tolerate the pain, but hopefully if someone knows enough to setup fail2ban, they should also be able to setup ssh config. Especially since reliable ssh connections is such a high quality of life improvement.

Yes I've hit this as well! Such an annoying behaviour.

However I think it's a good habit to make records in `~/.ssh/config` for each of your servers anyway just to keep tabs what, where, who, and with what keys.

> It keeps the logs cleaner.

I move my port somewhere else to reduce the clutter

> manually look at the logs every once in a while

I imagine that this is more by curiosity than anything else (which is a perfectly normal reason to do that - I do it from time to time just because). For logs analysis and alerting I have automated systems.

> using password auth is a bad idea

It is not if the password is correct, security speaking.

I disable password authentication and use fail2ban. It's unlikely they will be able to brute force my key, but no server is perfect. sshd might be compromised one day. I’d rather have an extra layer of defense just in case.

It's impossible they will brute force your key if you have a decent length.

While I'm sure it is possible for some (mainly government) actors to brute force keys, I'm also sure these do not include the same low-hanging-fruit vandals blasting brute force attacks. And I'm also pretty sure you're not one of the select targets of these highly advanced actors.

A vulnerability in sshd is indeed possible and happens once in a while. Fail2Ban won't stop this though because a known exploit will let them through on the first attempt.

I personally view fail2ban more as nuisance control when it comes to SSH with password auth disabled. Minimizing the log crap, the wasted CPU resources by the failed handshakes. It's not really a security protection in that scenario. In other cases (e.g. web logins where passwords must be used) it of course is.

Unless the maintainer of your distro's ssh package accidentally introduces an error that reduces the number of possible keys to, say - 32,767 total possible keys.[1] That's a brute-forcible number of keys that fail2ban would help mitigate.

(1: https://research.swtch.com/openssl)

Or the NSA subverts a cryptographic standard in order to produce predictable seeds for cryptographic random-number generators that are used to produce private keys. [1]

(1: https://www.bbc.com/news/technology-24048343 )

INCONCEIVABLE!

You keep using that word. I do not think it means what you think it means.

20 years ago port-knocking was supposed to solve this issue for good but it seems to have been never really been taken up. I'm not sure why.

Because it doesn't solve any real problems if you have ssh password login disabled and filter out login failures from logs

With port knocking or a simple as having SSH on a non-standard port, the connection request stops before even opening the TCP connection. That's less load on the system, less logs, less writing to storage, etc. Less of what you don't want must surely be preferable.

…all for the cost of vastly increased complexity. You’re just micromanaging your pets there. If the load of failed login attempts or the log writes brings your servers down, you have a whole lot of other problems to take care of.

Are we talking about ssh on an ESP32 or a 2U server? You'd saturate my transit port before I noticed the load on my server or the logs that I filter by default.

btw TCP is cheap compared to public key crypto

This is really only effective for password based logins, but most logins these days are key based

No, it cuts the DOS resource usage by a factor of 4 because a TCP connection is never opened.

Port knocking as it's usually done is easily sniffed. Perhaps using a dynamic TOTP-like time based seed to constantly rotate the ports might help. But it sounds overly complex.

It feels very like a "key under the third plant on the right" kinda thing. Not a solid security measure.

Single packet authorization. It's like the server is not even there unless you send a cryptographically signed packet

https://cipherdyne.org/fwknop/docs/SPA.html

Nice solution. I hadn't heard of it. I immediately thought of replay attacks because it's a one-way protocol but it looks like they mitigated those as well.

Moxie Marlinspike made something to address the simple knock server's problems.

https://archive.is/MZKkb

This is fine, though. "Security through obscurity is not security" but moving your SSH port to something not 22 will utterly eliminate brute force attacks.

It's too much bother to go find it, and the bozos will just move on to the next machine with port 22 open.

That's still not really security but just a nuisance mitigation IMO :)

It's both, really. If you're not getting scanned, you've reduced an attack surface, and that can only be good.

Well, I view it as hiding an attack surface. It's still there, just harder to find.

But I know I'm a bit of an absolutist on security.

You put a lock on your bike.

But you also put it in the shed, and lock the shed.

Nuisance mitigations are part of security too! Fewer irrelevant notifications makes it more likely you’ll notice when something really is a problem.

It’s like how an adversary might launch a DDoS attack at the same time as they exploit a SQL injection vulnerability to exfiltrate credit card information. Filling up logs and alerts overwhelms the blue team and makes it harder to notice the quieter, but more dangerous attack.

Security through obscurity actually is security, and is perfectly valid to use with a defense in depth strategy. The problem is when obscurity is the only defense.

I agree, these attacks are looking for systems that have pretty default security, and by running on a different port you avoid all this automated chaos because you're non default now. Like scam emails with typos, a way to filter out the naïve people.

> This. I really do not understand why people use fail2ban when the threat is somewhere else.

Fail2ban keeps my log short enough that I can review them daily, I don't have to sift through thousands of login attempt.

> It won't stop a ddos but will certainly, at some point, prevent you from logging in.

Yup, losing my key and having no password access will do that.

I'm not sure manual log scanning is all that interesting. You probably want to produce a list of successful logins and review that regularly.

If an attacker manages a succesful login you've already lost. Better to catch them in the act while they're scoping you out.

If your SSH key leaks you're not going to have a warning. All you'll see is a login from yourself that you don't remember.

I am pretty sure we turned off password authentication like 10 posts up this thread.

This is why I require both a private key and a password.

I have fail2ban configured to block IPs with invalid private keys after a couple attempts, and if the key is valid to email me and rate limit invalid password attempts.

This gives a more than sufficient warning if my key leaks which is already very unlikely, and this just makes it much more unlikely for both to be compromised, and only took an extra 5 minutes to configure.

How do you configure emails on successful logins? Can you share your config, sufficiently anonymized?

I created a jail for fail2ban with

  logpath = /var/log/auth.log

and for the filter I use

  failregex = .*Connection closed by authenticating user [a-z_]([a-z0-9_-]{0,31}|[a-z0-9_-]{0,30}\$) <HOST> port [0-9]* \[preauth\]

and the emails are just cron with a python script that checks /var/log/fail2ban.log for any new Found|Ban|Unban IPs and sends them using smtplib

If you like I can share the full config files but the rest isn't too interesting nor different from what is here https://www.fail2ban.org/wiki/index.php/MANUAL_0_8#Jails

I move my port to an uninteresting place and it considerably reduces the amount of logs. If this was a nuisance I would filter out the messages

> Yup, losing my key and having no password access will do that.

You can also lose your password. Or forget it if you know it by heart. But in any case you can use a password instead of a key - it just needs to be good enough (= long and not in cracking dictionaries)

I really don't understand why people use keys when the threat is somewhere else.

Assuming that your comment is serious, where is the threat in your opinion?

Bad passwords.

Then I do not understand. Keys are the answer to bad passwords (among other things)

It is one answer. Good passwords is another.

I prefer libshield but one thing I've found is that annoyingly sshd didn't use PAM to check if the login user is valid apparently (so it never fires when only using keys).

Not really. A lot of these bots are super duper dumb and continue slamming your server with handshake attempts even if password auth isn't even turned on. Every handshake is wasted CPU resources (and asymmetric crypto isn't cheap). It also makes it harder to see real dangers in the logs.

And/or run sshd exposed only to a wireguard or tailscale interface.

Not for other services... I have fail2ban parsing my NVR logs. Wrong password three times and it permanently blocks the IP on my Opnsense firewall.

And change the sshd port. I use my birth year. 0 brute force attempts in the logs after that.

Wow that's a cool evolutionary algorithm in practice! I'm actually surprised it worked, especially with the amount of fuzzing I'd expect ssh ecosystem to receive (well perhaps the servers do, but not the clients haha).

Distributed valgrind

Details!

Seriously this is both hilarious and intriguing and deserves a long form blog post or something.

Seriously, this should be standard practice. Fail2evil...