Secure your MongoDB, Redis, etc | Hacker News Reader

Secure your MongoDB, Redis, etc

medium.com

88 pointsshahinism45 comments

Having been the first 'DevOps' guy to hit the ground in a few startups I have seen interesting deployment and access strategies to databases PHP exec("echo $mysql_query | mysql -uadmin -ppassword"); is probably my favourite. For a lot of frontend or fullstack teams, the business shouting, get it done, trumps any damage they might take by a compromise. Also often they are just not aware of how insecure the defaults are, I feel like a lot more can be done to provide secure out of the box stacks for 'full stack' developers.

Probably. Feel free to provide concrete examples.

---

FYI, there isn't a security vulnerability in that wretched example... just WTFibility.

(I'm assuming shell escaping...else virtually every query wouldn't work.)

I'm staring down the barrel of a startup that "deploys" using an Internet accessible, passwordless Samba share to drag and drop PHP files from desktops to production.

Nothing surprises me.

Is this the result of the no-ops movement? Did people forget how to deploy software? I find it hard to believe somebody would run a database on publicly accessible ip/port.

For some reason redis thinks it's a good idea to listen on all interfaces by DEFAULT so you can perhaps understand why this happens. But I mean come on, what year is this. How often do we need to repeat that you need to use SECURE DEFAULTS!

Short of that I find redis an nice piece of software, I only wish it was somewhat more sane in its default configuration.

In theory I totally agree with the concept of using secure defaults.

In practice, however, I have seen far too many developers spend hour after hour trying to unwind a byzantine labyrinth of iptables and strange network configurations, weird firewall rules and corporate policies, never mind mangled DNS, to think that the correct solution is merely "secure defaults".

At the end of the day, developers get paid to "make it work" and unfortunately that strongly incentivizes the "open by default" configuration mindset.

Maybe developers should get paid "not to put us in the headlines with 1 billion accounts breached".

Maybe developers should care about not having their companies in the headlines instead of just pushing things out as quickly as possible.

Most developers past a certain seniority level know better than to run on a production database on a public interface without at least setting up good authentication. I think in many cases, it's just an issue of something slipping through the cracks.

A few example scenarios where this can happen: one may open a port temporarily in testing before something goes live and forget to close it back up. Legacy applications may be dependent on certain services at certain ports and it may not be trivial to change those, or even add authentication to them. The whitelist rules allowing communication can get mangled or misconfigured, especially if the legacy application's IP jumps around; in frustration, an admin may set the rules too wide to try to encompass all potential IPs, assuming that no one in the same /16 is going to try to break in soon and they'll come back and fix this in like a week or something. One may spin up a test instance into a wide-open security group and assume that someone else is going to come through and sweep it up, and before you know it that test instance is hosting real data and no one has tried to clamp it down, or that firewall rules for that port are not necessary because the authentication/whitelisting will occur on the software side, and then fail to set that up correctly, or run an insecure version of the software.

There's that old saying: "A person isn't stupid; people are". The constraints of the business often make it so that compromises have to be made and what we'd consider a "good solution" is politically intractable. No one likes that, but it's the environment many of us have to work in, like it or not.

All we can do is scream at the wall until at least one exec goes through a traumatic security event. It usually takes a significant breach to scare the exec team straight.

redis used to listen to all interfaces. The current stable release has fixed that. http://download.redis.io/redis-stable/redis.conf

Redis's "Protected Mode" was introduced a couple of years back iirc. It refuses to listen on anything besides localhost if you haven't configured authentication, unless you explicitly disable the protected mode flag.

If you're using the version of redis packaged in trusty/universe, it is extremely old and doesn't contain this. It also contains a handful of critical security vulns that are being openly exploited in the wild. Update manually TODAY.

The default in Redis changed, as a result of active exploitation:

https://kevinchen.co/blog/postmortem-server-compromised/

this is (maybe) because deployment is done by front-end engineers. don't get me wrong here, "lazy devops" is great and combined with cloud is just amazing (proc files, package files, ci files etc)

i don't think people want run the database or the cache system or any other critical service as public, but probably the background is not enough to solve issues on production when they pops up so:

1. you deploy a service over cloud 2. everything works 3. but if something must be solved at infrastructure level 4. apply('old style solution') open everything and debug from home attached to production

I think it's unfair to blame this on no-ops. People have found unsecured MySQL ports on the open internet for years dating back to the first LAMP stack deployments. And things like iSCSI targets, SMB shares, etc.

bad-ops is of course also a thing, and mostly comes from the same mindset ("how hard can it be?", and "why do I pay so much for this stuff?"), but I personally believe that the concept of no-ops is hilarious. Treating infrastructure as code is fantastic, but developers and ops people have different priorities, skillsets and approaches. Neither are dispensable in any kind of serious environment.

Well, no ops is not really no ops, but more "Google/Amazon is paying ops people for me".

Two years ago I was looking at rsync shares - still thousands of them out there, unsecured:

https://blog.steve.fi/Secure_your_rsync_shares__please_.html

There's still a lot of rsync out there (~160,000): https://www.shodan.io/search?query=rsyncd

> Did people forget how to deploy software?

People never knew how to deploy software.

Are there even guides to learn that? Seems not.

Your database should not run on a server that has a public ip. Forget authentication (it's plaintext on redis), or binding rules, or using a uncommon port.

It's network isolation or you will be owned.

Tell this to Heroku whose Postgres instances all are publicly accessible (TCP-wise) outside of their network.

Yes, why not tell this to Heroku?

Have a read of the Internet Security Exposure report by @binaryedgeio, there is a section regarding Data Storage, mentioning Mongo, Redis, ElasticSearch, and some other. In Summary, it was found 627.7 TB of Data Exposed.

http://blog.binaryedge.io/2016/10/07/internet-security-expos...

In this day and age with breaches happening daily and publicly why hasn't mongo taken security seriously and enabled authentication by default?

Maybe that would hurt their "get started easily" proposition, which was very imporant to get them where they are now.

yeah but now they're there, with a big built in base of people who don't want to stop selling their skills. So they can make a new get started securely proposition.

But it's always "grow or die"

This might be useful https://docs.mongodb.com/v3.2/administration/security-checkl...

(But let's say binding on 0.0.0.0 or not having a firewall in the system are not only their fault)

In the same way people can trust local addresses in Pgsql

Or why is it so hard to enable authentication at all?

Or why enable flags that allow to loose data without warnings in many common circumstances?

Don't expect anything from mongo. Don't trust mongo. Don't use mongo.

They have improved a few things. The new versions of MongoDB don't bind to the public interface and they have the capacity for authentication (unlike the early releases). I believe this is why the most popular versions of unauthenticated MongoDB are the 2.x versions:

https://blog.shodan.io/its-still-the-data-stupid/

Is using spiped viable for redis connections? Can all the servers that need to connect to the redis server all connect via spiped using the same secret key?

What about data in motion?

Is it viable to use an SSH tunnel with each service listening on localhost alone and using an encrypted tunnel to send data?

That's what the firewall is for. I set up iptables rules on all machines that only allow fixed IPs on the local network access to specific ports.

If you're running it over the public 'net, then I think Mongo and Redis allow for username/passwords and/or encrypted transit.

No it's not. It's secure but not reliable, your ssh tunnel will disconnect at some point. Use firewall and/or native security.

Thanks! I read somewhere that this was a good way to secure redis for compliances. Good to know it may not always work.

The main problem is DigitalOcean does not offer a centralized firewall, so they assume users will and can setup iptables. If they had a centralized firewall and default policy of all inbound traffic blocked and then you specifically open ports on droplets it would make a huge difference.

Ok. Repeat after me, "I will not use Digital Ocean for production. They are not a real hosting company."

The why is left as an exercise for the reader.

Use Digital Ocean for simple production use cases. They are a real hosting company, they are affordable and they are simple to use.

Professional non-trivial usage should be on AWS/Google/Azure/SoftLayer.

Bzzzt! Wrong answer, try again. They are not good for anything remotely resembling production. Since your production should try and closely resemble everything else I'd say they are not good for anything at all. The tag line is "Cloud computing, designed for developers." They should be even more careful about good practices and sound design keeping in mind the kind of customer they are trying to get. Over the years they have done anything but that.

They are perfectly fine for anything production resembling a guy that wants to run 2 servers on the internet.

And they're certainly better than trying to run that on someone's home DSL connection.

Actually your DSL is probably more secure because it wasn't designed by complete muppets. Who thought it a good idea to put everyone on flat Layer 2 network. Remember the time they were handing out block devices without scrubbing. How is the new Ceph backed block service. What's the p50 latency? What about p90?

Could you give a few details as to why?

I run my personal server on DigitalOcean and it has been pretty good. At work we run a series of tier-2 services on DO and it's been great for our use cases. We used to run a cluster of 100 machines there and it was stable and cheap.

I realise it's very simple, and that isn't good for a lot of production services, but I don't understand the hate?

stunnel. It's an incredible piece of software.

Do you have any thoughts / experience compared to spiped or ipsec? The author or Redis likes spiped: https://redis.io/topics/encryption

That's an excellent question. I've only ever used stunnel and I was blown away by how many different things it can do / ways it can work. It does so much for such a simple tool. antirez probably likes spiped because it does fewer things. :)

Reply on news.ycombinator.com