I discovered thousands of open databases on AWS
infosecwriteups.com
infosecwriteups.com
Even if you forget that the cloud is the internet and that the entire internet can reach you over the internet, it doesn't take a genius to set up a password for a cloud service. I have no idea how many of these databases are honey pots, but finding open and vulnerable servers is depressingly simple.
I suppose it's kind of liberating to know that you can be dumb enough to fling patient data into an unprotected cloud server and still get a job in IT. The bar is really set that low.
Well, managers aren't scientists looking for truth. They are people running businesses that need to stay afloat and get stuff done. There's a lot of wiggle room there, mostly due to time pressure (from a charitable perspective, less charitable: also because of company politics).
With so many 10x full-stack developers who're just looking to get sh done and clock out, it can't be otherwise. If managing a server becomes just another bullet point in a developer job, this is exactly what we get.
Sure I can deploy stuff, but you shouldn't trust me. Unfortunately many are not so honest or even aware of the complexity (unknown unknowns).
Yup...the same management are the ones that laid off the DBAs, system admins, network engineers, etc because "the cloud" hand waves all this stuff away now.
On the other hand, I would argue that there are way too many footguns hidden everywhere, like the default binding of docker mentioned here. So yes, a highly skilled IT security professional knows them all, but they are rare. And I think it should be possible to set things up securely in a straight forward way, for people with their speciality in other areas, of which there are plenty.
If networking 101 is a foot gun to you, I don't think you know enough to be responsible for anything important. I don't think there is a software fix for that lack of knowledge.
I didnt need to get beyond there to see when you went wrong, and the bar for competence is not high across the board. Sadly I havent found a way to make money out of it, but you have to be aware of it to navigate the world safely.
There are two things at play here:
The reality of what happens is that devs with limited experience are asked to do things far outside their comfort zone because they aren't staffed enough. This isn't on the dev, it's on the company and a reality when it comes to growth. People make mistakes.
The second thing is that the databases themselves should be secure by default but are not. That's on the developers of that software. DBs should require setting strong passwords on creation.
Aws should also warn users about exposing these services directly to the outside world instead of just within the vpc.
It's as simple as that. And at scale, it adds up to a lot of mistakes.
And for better or worse, much of the tech industry does not require extensive certification and training with checking and rechecking of every step. (And even aerospace makes mistakes.)
And, yes, junior people make even more mistakes. But I'm not sure the fix to that is to have junior people's every action be check and rechecked by a more senior person. It can be done. That's generally how things work in medicine for example. Along with stringent certifications. But I'm not sure how many people in tech would want that. It's a reasonable discussion to have. Maybe training wheels are taken off too early in many cases.
Where are you going to store the password?
I'm not saying it's rocket science but it does take consideration. Do you store it in a plain text file on the server? How do you deal with CI/CD? Do you run a separate service for one credential?
AWS itself has a Secrets Manager from which other authorized services can pull secrets. If it's an outside CI/CD platform then those usually also have a place to store credentials.
Don't get me wrong. I don't think anyone should skip this step. You need auth for your datastores.
Binding to 0.0.0.0 is unfortunately the default for Docker. I wish it would have been different.
I accidentally opened up a Redis test-instance to the world like this.
Assuming I add a rule via
`iptables -I INPUT -s 123.123.123.123 -p tcp --dport 8080 -j DROP`
and on interface:port 0.0.0.0:8080 of the host is a container listening which got run as
`docker run -d -p 0.0.0.0:8080:8080 some/server:latest`
Will that container still be accessible to 123.123.123.132?
I was bit by this once. Be very careful with docker. Do not rely blindly on your firewall of it runs on the same machine.
So, if you're doing any filtering for internal network addresses == internal traffic, congratulations, now your inbound external IPV6 traffic is recognized as internal.
I'm not using --iptables=false but I'm not discounting the possibility that maybe I've configured this on a more global level somewhere.
I even follow this standard rule for pet projects
That isn't just a docker thing, though a few notable accidents with docker have shone a light on the issue in recent times, and is one of the reasons I much prefer to keep my firewall away from where things are running, between them and the unwashed masses on the public network.
Even at home I have a separate small firewall box between the server that hosts things (and the rest of my network) instead of trusting that host to manage its own security in that way. If I run a service via docker or anything else on there, the outside world doesn't see it unless I open a way in elsewhere. For extra paranoia, the network leg that others can connect to (wireless AP & wired network ports in the guest room) is also separate and only sees parts of the server(s) I explicitly want it to (you can do this with vlans and such depending on your kit, I've actually just got an extra NIC on the router).
Never assume a process/service/container/etc is at all locked down, unless you have checked it out yourself or have it sufficiently wrapped in things you have locked off.
Logical cloud protection layers - like security groups - are an additional security layer but not a replacement for proper route design.
This Google Search returned an extremely high number of mentions of various companies' ills, a high signal to noise ratio. It's very saddening to see that.
https://www.google.com/search?q=unprotected+databases+on+aws
With some even rudimentary knowledge of how IPV4 works this wouldn't be an issue on AWS, there are good defaults and tools to help avoiding this, no matter if you run Dockerized databases or bare metal dbs or use RDS.. you really have to explicitly open up your servers to end up open on the net..
For example if a developer needs to access a resource within a VPC then this sky rockets complexity. You can't just connect to something running in a VPC from the outside world. You'd have to install and configure a VPN and it becomes a whole ceremony for both the person setting all of this up and the person connecting.
Using a VPC/VPN and keeping your important things internal (like a database or staging environment, etc.) is a really good idea but unless you have someone who has a really solid understanding of this it's a whole lot easier to make something public, restrict a security group by IP and keep a whitelist of IPs up to date for developer access.
Whenever I read stories like these, it seems clear to me that someone moved to the cloud in order to not have to care about security. The 'cloud does everything for you!'. Just like you imply in your answer that PaaS, the next level of abstraction, will solve all your security problems. This move, however, will inevitably lead to a situation where people work with new and complex systems that they don't understand (remember: not having to understand them is the sole reason they use them). Unfortunately, working with complex systems you don't understand is the number one reason for vulnerabilities in the first place.
I am not convinced that a service exists that abstracts security away from you.
Zero trust everything.
I would trust Microsoft more on patches/configurations for an email system than something which is managed on premises. You need to have really good people to maintain a good level of security. Not only technically good, but also with a string cold management that will force updates even if it means the CEO will not get his maol for 15 minutes - and say that this is life and that the discussion i sover.
On top of that, MS would (I hope) install patches on their customer-facing systems in advance of an official patch release.
The above applies to the majority of large SaaS services.
Now when you have a "Platform", a hoster that requires you to bring in knowledge and not only data then it gets dangerous. You need to maintain the security of what you bring in. This can be an OS (your "Platform" provides VMs), or code (your "Platform" provides code runners). Unfortunately, when a company moves to the cloud, they sometimes forget to do this assessment and end up with monstrosities they installed themselves (which is not different, security wise, from having it on premises - augmente nu the 7B population that potentially has now access)
I would expect the latter to be sure by default.
Elasticsearch until recently did not nudge you to set up a username or password by default. I noticed the last time I installed it on a fresh instance that on completion of the install it gives you a warning about this and tells you what to do to set a password. That is a small improvement.
Most people would not have the service bound to a public interface, but for those who do for whatever reason have a set up where they are accessing it remotely, at least now there is a tip off that it is completely open to the world by default. This is different from pretty much any other service you might install. MariaDB for instance by default does not allow remote root login even if you change the config to bind it to 0.0.0.0.
A lot of people are just totally unaware of this issue. When I read about database leakage, I generally assume 90% probability it was elastic, these days. Defaults are so important.
If it's a test instance then this is a nice obvious place to go get it from. If it's not then the service isn't open by default, and in both cases we're not asking people to enable remote access themselves by some scheme which may allow them to do it without setting a password.
Same thing with TLS IMO: let certs be specified, or default to generating some - any option is better then default no encryption.
That exploit was single-handedly responsible for hundreds of thousands of bots (and more likely millions in later years) which wreaked havoc on the internet as a whole -- these were the days when a handful of bots on good connections could take down the Amazons of the world for lengthy periods (days, not hours). That they let it go on for so long was comical at the time, but in hindsight seems criminally negligent.
It cost that much because AWS is incredibly expensive.
If a product upon unboxing promptly flops on its back with "come here internet" access controls, even if by good fortune it's saved by your network ACLs, it's time to put it back in the box and return it.
Don't repaste your own comments just because there's been a dupe.
Also dupes are pretty common, it'd be weird to the repaste the same discussion into every dupe thread.
One additional consideration is that the secure defaults should be usable, otherwise the tendency will be to just disable them entirely (think SELinux and how commonly that's turned off as the first action on server setup)