Two main lessons for me:
1. Always run services (e.g. ElasticSearch) with a unique user dedicated to that service and nothing else.
2. You're never as secure as you think.
1. Always run services (e.g. ElasticSearch) with a unique user dedicated to that service and nothing else.
2. You're never as secure as you think.
Quick tip; If you do this, you can also make your iptables rules be per user. For example, "webserverUser" can only accept inbound connections on 80/443 and only have outbound connections that are related to established inbound ones. If an attacker gains execution as this user, they cannot download new code, etc or even do DNS lookups for that matter.
Instead of having 'nc -l 8080 | bash' or whatever as your payload, an attacker can just run code instead. "pwd > /var/www/html/exfiltration.html". If they absolutely need a shell, they could e.g. alter nginx or its config files to run `bash` on POSTs to a hidden route.
This does make it a little trickier, and potentially a little easier to detect. But it certainly doesn't make it so that "they cannot download new code".
Correct, I should have said, it eliminates many easy ways to download code. Defense is depth is all about making the attackers job harder and increasing their likelihood of being detected.
Good thing I don't do dev ops except for my personal projects.
It's simply reality. I don't enjoy it. Fortunately I've never done that in any type of production environment (just development). But I don't entirely blame people. Many of the infrastructure pieces that need to be deployed can get very complex very quickly.
I used to be in the same boat as you until I had my lightbulb moment: if you're constantly coming across permission errors like this, it's a sign that you're just doing things wrong.
I don't mean this as an insult; it's more that often when you download, install and provision a new piece of kit, it's easy to do so without taking a few minutes to read the docs and find out the best way of doing things. Especially if you're glancing over e.g. Ubuntu instructions when installing on CentOS.
(in fact, this rule applies to a lot of IT. Spending hours trying to get a CI tool to do X, Y and Z? Chances are it's just not meant to do it that way. God knows how much time I've lost to this.)
> I don't mean this as an insult; it's more that often when you download, install and provision a new piece of kit, it's easy to do so without taking a few minutes to read the docs and find out the best way of doing things. Especially if you're glancing over e.g. Ubuntu instructions when installing on CentOS.
As a sysadmin, I find software documentation to often be the worst place for deployment advice. It's often written by developers with the same mindset as the GP - just get it working, best-practices can come later (which they never do).
One app per user (and the corollary, proper segmentation between system users) is something you just have to train yourself to do, screw what the developer had in mind with his deployment instructions.
Though to be fair I've also run into weird bugs with OpenShift, Mongo and a few others over the years so it's not necessarily something wrong that the developer is doing.