Comment by snowwrestler | Hacker News Reader

Comment by snowwrestler | Hacker News Reader

Parent Full thread

Back when I started managing self-hosted sites, I would look through access logs as well. We even had an IDS for while that would aggregate the data and flag incoming attack attempts for us.

Eventually I stopped proactively reviewing logs and stopped paying for the IDS. It was a waste of time and a distraction.

It's not hard to find really useful content that summarizes common vulnerabilities and attacks, and just use that to guide your server management. There are a ton of best practices guides for any common web server technology. Just executing these best practices to 100% will put you way ahead of almost all attackers.

And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities.

Where logs are super helpful is in diagnosing problems after they happen. We used log analysis software to store and search logs and this was helpful 2-3 times to help find (and therefore address) the root cause of attacks that succeeded. (In every case it turned out to be a known vulnerability that we had been too slow to patch.)

> In every case it turned out to be a known vulnerability that we had been too slow to patch.

Yes. This is why relying on "patching" is a bound to fail at some point. Maybe it's a 0-day, or maybe the attackers are just quicker.

The solution to this is defence in depth, and it's very easy for most services, especially when self-hosting personal things. Few tips most people can do is.

Put up a firewall in front or put it behind VPN/tailscale.

Hide it in a subfolder. The automated attacks will go for /phpmyadmin/ , putting it in /mawer/phpmyadmin/ means 99.9% of the attackers won't find it. (This is sometimes called security by obscurity and people correctly say you should not rely on it, but as a additional layer it's very useful).

Sandbox the app, and isolate the server. If the attackers get in, make it hard or impossible for them to get anywhere else.

Keep logs, they allow you to check if and how you got attacked, if the attack succeeded and so on.

Depending on the service, pick one or more of these. Add more as necessary.

The key thing is that you should not rely on any ONE defence, be it keeping it patched or firewalled, because they will all fail at some point.

> This is sometimes called security by obscurity and people correctly say you should not rely on it, but as an additional layer it's very useful.

Anyone who thinks “security by obscurity” is useless should try reverse engineering some properly obfuscated executables (or even code). Obscurity is absolutely useful; definitely not a complete solution by itself, but a very useful component to a security solution.

The term "security by obscurity" or "security through obscurity" implies that ONLY obscurity is being used to provide the security in question. Like leaving a totally unsecured server on your network with root access available with no password, and telnet or sshd open on a high numbered port instead of the regular ones.

Obscurity is a useful tool to be added on top of real security, and can help reduce the random baseline doorknob jiggling attacks, where people are just scanning the standard ports. But obscurity by itself is not enough to provide any real security beyond that.

You are completely correct. For a level 3 person.

Level 1 : I don't know what I'm doing, so I'll invent stuff only I know in the expectation that'll be enough. This person is told (with good reason) that security by obscurity is no security at all.

Level 2. They got the above message, so do everything right. Setups, firewalls, permissions, and so on. They are proud of their expertise and lecture level 1s all day long.

Level 3. Understand that all the fundamentals need to be done right. Add addional obscurity onto that because it doesn't hurt, and can filter out some useless traffic. (These folk should also lecture level 1 with the simplified message, but can explain the benefits to level 2.)

The problem with HN threads like these is that I don't know who's giving the advice. Level 1 2 or 3. Equally readers could be any level. Which might be dangerous if they are level 1.

IF you ARE level 1, learn the correct way to secure things first. THEN feel free to add obscurity onto that if you like.

I dunno, it feels to me that people are too ready to tell "white half-truths" to simplify the message, and assume everyone else is too dense to understand nuance...

Why can't we just outright say to everyone: "learn the correct way to secure things first. THEN feel free to add obscurity onto that if you like"? There's probably some way to convert that message into a catchy phrase, instead of just demonizing security by obscurity, yet having a small sect of elites in the know who break their own "rules"...

FWIW, this (rant) applies to a broader scope as well, beyond "security by obscurity".

When educating anyone about anything we tell half-truths all the time. It's a necessary path to understanding.

(For example, we teach kids the earth is round, then we tell them it's an oblate spheroid, then we tell them even that is a approximation. )

In a forum like this, the message is often distilled because one doesn't know the makeup of the audience. The simple rule is a good starting point for entry-levels.

Nuance can be hard to convey, because its usually a combination of context and resources which determine how far down a rabbit hole you can go.

I agree that this happens all the time, in every field. I'm dumb when it comes to my car, so my mechanic gives me simple rules to follow. Doctors tell everyone to eat less, exercise more, but in truth you can eat too little, and exercise too much.

The -real- problem happens even a simplification becomes obsolete. We see this in security a lot. Someone wrote the company policies 20 years ago, and they're insisting on say no-paste-on-the-password field, because that's in the policy, even though its since been shown to weaken security.

Yeah but why is it a saying and "security by anything-else is not good" isn't a saying.

What layer of security is sufficient on its own? None. Why is "obscurity" being singled out?

It always feels a little condescendant, like "oh I got a cool saying that applies to most noobs in security, let's use it", implying one is assuming the other party in the conversation knows less.

How would you apply this "anything else"? What "anything else" did you have in mind?

If you can come up with some good examples, maybe we can make that a thing.

Until then, we know that "Security by obscurity" is really bad, if that's the only thing you're relying on for security.

Otherwise, that would be "Security by something else plus obscurity", which might or might not be a bad thing, depending on what the "something else" happens to be.

Access lists alone are not sufficient Authentication alone is not sufficient IDS alone is not sufficient Encryption in transit alone is not sufficient Etc.

Reminds me of a friend that does security stuff. He says never store or transmit keys in a useful form. Bonus using munged keys will trigger alarms.

>security by obscurity

I detest this phrase right up there with "fake it till you make it". All security is by definition obscurity. Just a meaningless platitude that rhymes.

I suppose un-formalized un-proofed security practices will eventually be broken or counting on hackers not to do any investigation of your system will get you hacked, doesn't roll off the tongue though.

> All security is by definition obscurity.

Can you expound on this? For example, if I add a 30 second lockout after failed authentication attempts I don’t see how that comes under any non-tortured definition of “obscurity”.

The lockout doesn't exist without the authentication system in the first place. Which exists to keep people from knowing the information. Its a subset of something that is needed for obscuring the information in the system from anyone that should not see it.

I really feel dirty trying to justify this level of nerdy pedanticness. I'm sure you can poke some holes from my off the hip internet comment if you really want to I'm not trying to be academic. I mostly fueled this comment with my distaste for that other platitude.

I think you miss the point of security through obscurity. It's not about keeping the information itself obscure (in your example login information), but rather the method. For example, your password hashing mechanism. If you have a strong password hash function, you don't need to obscure which hash function you use (otherwise, open source software couldn't even exist in certain areas). However, if your security relies on you obfuscating your broken, home-made hash function that only hashes the first three letters of the password, you're not really secure. Security through obscurity is an attempt at securing an otherwise unsecure system by hiding or disguising the implementation.

That being said, obscuring parts of an otherwise secure system is fine as an additional layer, especially if you just want to deter script kiddies that always hammer the same endpoints

yes, also add cryptographic complexity, don't think that qualifies either.

Cryptographic complexity is math if you ask me?

A hammer is not construction, it's a tool used by construction?

"and if at all possible keep them off the public internet", this is the way. I would recommend going beyond a VPN to implement zero trust networking which does outbound-only connections so that its impossible to be subject to external network attacks. Tailscale does part of that, other exist, such as the open source project I work on - https://github.com/openziti

> And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities.

Just curious, do you leverage any tools to decide when to patch or is it time-interval based? We currently attempt[0] to update our packages quarterly but it would be nice to have a tool alert us of known vulnerabilities so we can take action on them immediately.

[0] "Attempt" meaning we can't always upgrade immediately if the latest version contains difficult-to-implement breaking changes or if it's a X.0.0 release that we don't yet trust

Besides pointing pentester tools like metasploit at yourself, there are some nice scanners out there. Examples in no particular order:

https://github.com/quay/clair

https://github.com/anchore/grype/

https://github.com/eliasgranderubio/dagda/

https://github.com/aquasecurity/trivy

So then you set up something like a cron job to scan everything for you and email the results once a week or whatever if you don't want to monitor things actively.

I just setup GVM / OpenVAS[0] to work out where I need to put in some maintenance work, how does that rate as worthwhile in comparison to those you've listed above? (which I will also look into).

(not a fan of the effort Greenbone have gone to for hiding their community edition and promoting their commercial products)

[0]: https://greenbone.github.io/docs/latest/22.4/container/index...

Thanks for these. I should have clarified, I'm more interested in something that will alert me when newly-discovered vulnerabilities surface. The systems I maintain are protected [enough] today but a new hack could drop that doesn't make mainstream media and I may not hear about it. We have annual security audits but it would be nice to patch things immediately. Aside from subscribing to a security forum/discord/slack, I'm wondering what other methods folks are employing to solve this.

Keep your pentesting tools up-to-date. Run them against yourself on every single deployment, if you can. Don't just run them quarterly because that's all that your PCI-DSS requirements say you have to do.

Integrate security code scanning tools into your CI/CD process. Tools like Dry Run Security, or something comparable.

There's much more, but that has to do with how to run your CI/CD systems and how to do your deployments in general, and less to do with security aspects thereof.

For Gentoo: glsa-check --list

The simplest is to only use packages from a distribution like Debian and run unattended-upgrades or equivalent for the security-updates repository. They usually fix vulnerabilities in less than a day.

I’m out of the self-hosting game now, but back in the day we just tried to keep up on security announcements.

Reply on news.ycombinator.com