How to Protect Your Infrastructure Against the Basic Attacker | Hacker News Reader

How to Protect Your Infrastructure Against the Basic Attacker

blog.mailgun.com

180 pointsbenarent39 comments

One thing you should probably do: establish a VPN. Give staff cert-based credentials to it. Block all inbound traffic to the VPN except a) port 22/etc to your bastion host and b) 80/443 to your front-end web boxes.

This greatly reduces your surface area, and helps prevent easy mistakes later (like leaving a staging server unpatched) from blowing up your whole deployment.

Also, admin applications: an easy way for smart teams to lose. These are a great candidate for "only accessible if you're VPNing in", which you can enforce at firewall and again in the application if you want.

Password managers; TFA everywhere but especially on corporate email accounts; ask yourself "If I were a hobbyist building a Bitcoin exchange where would I host it?" then don't host there.

Here's a useful but hard-to-implement recommendation: keep an up-to-date list of every box in use and terminate anything you find not on that list. Many compromises start with "The summer intern's project from last year was web accessible and..."

While i think this is a great idea. I just thought about the mess i would create setting up continuous deployment and therelike.

This fits my paranoia, but not my software environment :/

If you use an actual VPN, and not an SSH bastion host, continuous deployment works seamlessly. You can use a client like Viscosity to ensure that you're basically always connected to the VPN. In some ways, this setup is actually more convenient than the standard "SSH to each of the boxes" setup.

> ask yourself "If I were a hobbyist building a Bitcoin exchange where would I host it?" then don't host there.

What's the rationale for this? This would seem to rule out inexpensive VPS hosts like DigitalOcean, Vultr, and Linode, that are good not only for hobbyists, but for small companies on a shoestring budget. Are you saying something like AWS is better simply because it's less attractive to hobbyists?

Linode just had yet another major compromise. Yes, it should probably be ruled out.

Don't ever use Linode. If you have friends using Linode, tell them to stop.

Just went to cancel a VPS I have over there... There's so hosed with the DDOS, I can't even bring up the homepage!

I really think it's time for guides to be written that don't contain the advice to disable all IPv6 support. IPv6 is coming. It's 2015 and categorically disabling (or actually just breaking) IPv6 definitely isn't good advice any more.

Yes. The article did qualify that advice with "unless you're using it...", but people should be doing v6, so advice should include v6 too.

That. I was relived when I saw he advising against disabling ICMP, just to read that at the next paragraph.

It's past time ops security realize their job is to make the system run securely, not to stop it so nobody gets in.

This. At least, if you're not ready to accept incoming v6 connections (which really isn't hard to do), use connection tracking to let outbound v6 through and add the necessary icmpv6 accept rules.

ufw comes pre-installed on ubuntu and is dead simple to use, there's really no reason not to use it.

  # ufw allow 22/tcp
  # ufw enable

should be all you need to have connection tracking on both v6 and v4, have a tried and trusted icmpv6 accept list, and keep your v6 and v4 firewalls in sync.

I like to use a limit rule (ufw limit 22/tcp) for SSH, to discourage brute force attacks. If nothing else, it will help reduce the noise in the log files.

This isn't a replacement for basic precautions such as disabling root login and not allowing password authentication, of course.

I prefer to not limit SSH but use fail2ban instead to remove the noise.

It's even 2016 now.

This is a decent "basic" guide. I look forward to the "intermediate" and "advanced" versions. Hopefully, Mailgun will come through and actually deliver those in the near future.

Author of the guide here, don't worry, we will!

You should also tweak your guide for RHEL7 based distros. Things like journald, firewalld etc. Also remember that in RHEL7 sysctl and security.conf are configured a little bit different now.

Red Hat Enterprise Linux 7 Security Guide is a good reference.

If possible better both common Linux variants: RHEL/CentOS and Debian/Ubuntu.

Thanks for the reassurance. I've read too many of these "part 1" articles promising follow-ups that never transpired.

As somebody who has written some "part 1" articles that promised follow-ups, sometimes the negative comments on the first article discouraged me from writing part 2. Useful, constructive feedback can help fix the article and feed into part 2, but kneejerk negativity about something you've done for free is hard to hear. Even more useful would be someone offering to help write part 2.

> ... sometimes the negative comments on the first article discouraged me from writing part 2.

I certainly understand that and I hope I didn't come off sounding ungrateful. A few years ago, I spent a lot of time writing articles and recording videos for my blog and, fortunately, I never had to deal with such negative feedback. I can certainly see how it would discourage you from continuing.

It's simply a bit disappointing sometimes to Google for something, find an article that sounds like exactly what I was looking for, discover it's a "part 1" that didn't quite cover what I needed, then go looking for "part 2" and realize it was never written.

It is also often the cast that best intentions lead one to write part 1 but life gets in the way for part 2. I try to tune out feedback that isn't constructive (like a lot of feedback is these days). I wonder if it is (or could become) a thing to ask the original author if they'd mind someone else taking up a new draft?

I don't mind the criticism constructive or otherwise.

I have already sketched out where to take part 2 and filled in some of the sections. That being said the 80/20 rule applies here like everywhere else.

That being said, if anyone wants to collaborate on part 2, feel free to ping me at rjones@mailgunhq.com and we make a mailing list.

Thanks for the very useful article. Minute point: for Rackspace, I thought that ServiceNet is a shared network and you require Cloud Networks to establish a private net.

Simplified, generic, basic guide follows. You can probably find a guide for hardening all these individual things on your system, so trying to enumerate them all would be a losing game.

Primary question to ask yourself: What connections are allowed from the internet into each host, protocol and port on your network?

Primary task: Firewall off all traffic coming into your servers from the internet to be only web and VPN access.

Secondary question to ask yourself: What stuff can run on my servers, and by whom, and what access do those users have?

Secondary task: Limit the users who can run programs, limit what parts of the system those users can view or modify, limit the programs that can run, limit the files and directories that can be accessed.

Tertiary question to ask yourself: Is my software full of bugs or security flaws?

Tertiary task: Use software designed to be secure by default, configure it to be secure, and update it for security patches constantly. Note that this does not mean "upgrade it constantly".

Additional considerations: Don't use shared accounts, don't use root, use keys/certificates whenever possible, use a separate machine for the VPN, put some sort of network intrusion detection/firewall/defense-in-depth/blah blah network appliance in front of the public facing servers, use a web application firewall, monitor your logs for unusual behavior, and get someone who's very familiar with security to double-check your setup (read: break into your servers and tell you the holes they found)

"This guide will eventually have three versions, Basic, Intermediate, and Advanced, with each version focused on defending your infrastructure against a different class of attacker." Love this approach! It must be a different game defending against really talented and dedicated people vs. a bot. I'm looking forward to parts 2 & 3. Those little "DO" and "DON'T" bullet points were a good idea too.

Thanks, the DO's and DON'TS were heavily inspired by a talk given by Colin Percival called "Everything you need to know about cryptography in 1 hour". You can check it out here: https://www.bsdcan.org/2010/schedule/attachments/135_crypto1...

It's really good!

russjones is the author of this piece.

Thanks for contributing to my massive backlog of "talks I really want to watch but don't have time to" :)

wow Keyczar got it wrong ?

funny the whole purpose of this project was to have cryptography setup with the right options

Is there a video/audio of this preso somewhere ?

edit: found it

https://www.youtube.com/watch?v=jzY3m5Kv7Y8

Very nice article, even for advanced users (not attackers ;)).

I have one question though. What are your thoughts on DROP vs REJECT firewall rules, as some people claim that DROP offers no additional benefits over REJECT while causing inconvenience to legit users. ref: http://www.chiark.greenend.org.uk/~peterb/network/drop-vs-re...

I suspect reject uses some bandwidth and CPU time.

Shouldn't it be the opposite? If clients receive a REJECT message, they will simply stop trying to connect, but if they don't receive any response at all, i.e., DROP, they will keep on trying to connect before any timeout threshold kicks in, thus putting more load on the server.

Thanks for the guide, very useful. Spotted a very minor typo in the Backups section: "Backups server two primary purposes" (should it be serve?)

Fixed that, thanks for pointing it out.

I'm surprised the article doesn't recommend using ufw or firewalld to set up the firewall. I like ufw, which is available on Ubuntu and Debian at least. Also, what about fail2ban, for automatically banning IPs that repeatedly attempt unauthorized access?

If you are using public-key cryptography (instead of password) for SSH there isn't any need to ban failed attempts.

I recommend “Securing Debian”:

https://www.debian.org/doc/manuals/securing-debian-howto/

Every once in a while, you find a jewel in the sea of crappy security guides. This is a jewel: go for it.

Just say no to VPNs.

Reply on news.ycombinator.com