Hacker News Reader

Comment by tptacek | Hacker News Reader

This is a vulnerability in the authentication scheme used in the MySQL wire protocol, meaning attackers need to be able to connect to your MySQL database directly to exploit it. Attackers should never, ever be able to connect directly to your MySQL database directly. If you can connect to your MySQL instance directly from your Macbook in your living room, fix it right now.

View on HN

―Attackers should never, ever be able to connect directly to your MySQL database directly.

mySQLgame[1] demonstrates that domain logic can be successfully implemented within a publicly accessible database[2]. It would be better to reword your statement to:

Minimise the attack surface by preventing unnecessary access

There are times where public access to a database server make perfect sense. It is the reason why database servers implement TLS client certificate verification, Role Based Access Control (RBAC) and other security features which a typical application (with domain logic at the application layer) has no hope of implementing correctly.

[1] http://mysqlgame.com

[2] http://martinfowler.com/articles/dblogic.html

I would generally avoid building applications that assume clients are going to speak directly to the database. We see a couple of them every year (it's a common pattern in enterprise applications) and they tend to be horrorshows.

In any case, the typical web app deployed by HN readers has no business having an exposed MySQL port.

Actually that's exactly what we do with LedgerSMB. The database is the final enforcer of security matters and the authoritative voice in authentication (the web app logs into the db with user-supplied credentials). A lot of traditional middleware functions are pushed into the db (as stored procedures) where these can be reasonably represented as set operations against the database. As an ERP application, this means that this includes just about everything except generating printable invoices or HTML.

This works well. There are some things we are still working on improving, but it's getting there. Moreover it means multiple clients on multiple codebases are possible and we don't have to worry as much about the implications of what happens when someone writes a secondary client to hit the database.

At the same time, we use PostgreSQL, which I have a bit more confidence in than MySQL. Of course I would still suggest limiting access only to those IP address ranges where that is necessary.

Oh god you poor man. We took a look at using SMB Ledger (the parent of LedgerSMB) years back and after peeling back the covers it just looked horrific both from a security and general code quality perspective. This was around the time of the fork, and I couldn't help feel that SMB Ledger had it's work cut out. Hopefully you guys have been able to get through to the other side.

Well, here's the status so far.

1) Going to a db-centric security model has allowed us not to trust the SQL-Ledger code (good thing too).

2) We are refactoring/removing SQL-Ledger code as quickly as possible. As of 1.3 this means payment logic, reconciliation logic, contact management logic and more. 1.4 will hopefully rip out and replace all search and reporting functions. It will take us a few more years to get the codebase where we want it though.

3) The bad thing about bad code is that bad code is contagious. When you spend a lot of your time debugging bad code it is very hard to write good code. Most of what we wrote for 1.3 will need to be rewritten again. I am pretty happy with the code we are writing for 1.4.....

I think we have come through the worst of it. Pace of development is speeding up which is a good sign and much more of the application is subjected to unit tests.

As a side note, when we first added unit tests to the number rounding tests for the code we inherited from SQL-Ledger, there were failures. We replaced that logic very quickly.

Yeah, it was pretty bad... Now its getting better.

Edit: Also it seems to me the worst never seems so bad when you are in it. I don't think that I could see how many problems we had from this until I am here, half way from 1.3 to 1.4, asking why 1.3 took five years to release (we beat Perl 6 and HURD though, I guess Duke Nukem Forever in fact beat us date-wise by a couple months).

Looking back at the customers of mine who have had problems, or projects that went way over budget or took too long because of difficulties here, I can see how much that hurt us. At the time though, it was just one of those keep working on it kind of things.

Either use iptables to lock it down to a particular IP or set of IPs. Or setup ssh tunnelling to expose a remote mysql server as a local port. You should assume mysql is insecure and needs protecting.

You're just swapping one exposed service for another in that case. Ie, OpenSSH instead of MySQL.

On other other hand, I'd trust OpenSSH more than MySQL.

Well, you're reducing the exposed services from two (mysql + ssh), to one (ssh). Which is always a good idea.

Also agreed. I'd trust OpenSSH to do security better than MySQL.

Having OpenSSH running is pretty much essential for any machines you remote administer. Some things you can do: disable password auth, use public keys. Disallow root logins. Listen on a non-standard port. Configure a hardened "jumphost" as your interface between your machines and the outside world.

Only if you make the assumption that SSH is already exposed. Which most of the time, it doesn't need to be.

Also: SSH is considerably more battle-tested in this configuration. There's a lot to be said for being aligned with how the developers imagine a program is used

The counter argument being that if you expose lots of MySQL servers directly to the Internet, you end up with a product much more suitable to being exposed to the Internet.

"Never" ... You are aware of the existence and mass use of shared web hosting systems right?

Don't use shared web hosting.

If ever a comment deserved down voting into oblivion, this would be it.

Your advice is for everyone that is providing and using shared web hosting, to stop it?

From a security perspective it's good advice. I've never seen a shared web hosting provider whose boxes couldn't be owned by an account that could drop code and execute. Local priv escalation holes seem to be a dime a dozen and you don't know if they've even attempted to lock down their users' data.

Now, do you really care if some community bulletin board's database gets owned? Probably not. But I wouldn't run a shopping cart on a shared hoster.

If you don't really care about security --- and I agree there are times when you shouldn't --- then by all means use the cheapest possible hosting option available to you. But if you care even a little about security, avoid shared hosting.

I really don't understand Mike Cardwell's objection; I don't think what I'm saying is controversial at all. I actually thought I was making a relatively banal point.

I'll just leave the following here. Maybe you can figure out from it what my point was:

You: Attackers should never, ever be able to connect directly to your MySQL database directly

Me: "Never" ... You are aware of the existence and mass use of shared web hosting systems right?

I think you're having trouble with the intended target of the word "you" in my comment. I'm not writing to people hosting Minecraft forums on Dreamhost.

Perhaps you should have been more careful with your wording. It demonstrated a lack of understanding of real world configurations and requirements and implied that if you're doing it that way, you're doing it wrong. I'd guess that most websites live in shared hosting systems.

EDIT: You could have just replied to my original comment agreeing with me that shared hosting systems work that way, and that it's ok for certain types of site. It would have made more sense than your comment "Don't use shared web hosting."

I'm really not sure what you're hoping to have me concede here. If you operate the kind of application that people on HN tend to operate, you should avoid shared hosting. I work with and enjoy talking to people who are serious about running applications, and I provide advice to people who are at least somewhat serious about security.

If you don't fit either of those molds, I don't think any less of you, but I'm not going to tailor my advice to you either.

It really sounds like you're just looking for something to be indignant about. I don't know you or anything about you, so I had no expectation that you were that kind of person. Consider addressing your objections to the thread, instead of aiming them at me, if you'd like to avoid that appearance. For truly, I do not care whether you like shared hosting or your friends are struggling indie shared hosting operators. That's not relevant to me even a little.

A less personal way to frame your objection, rather than "Are you commenting just so you can be downvoted to oblivion", would be to write a comment that starts with the words "There is another side to this that readers should consider..." and go from there.

You've already conceded the point by completely rewriting your claim.

"If you operate the kind of application that people on HN tend to operate, you should avoid shared hosting."

Is a far cry from:

"Attackers should never, ever be able to connect directly to your MySQL database directly"

That comment was about as useful as:

"Attackers should never, ever be able to enter the data center where your servers are hosted"

"Now, do you really care if some community bulletin board's database gets owned? Probably not. But I wouldn't run a shopping cart on a shared hoster."

I agree. tptacek is the one who disagrees with you. He thinks there is "never" a good reason to use shared hosting.

If you're using it for a real application, I'd move.

Sorry if that makes you feel bad.

I'm not using shared hosting, no. Although I did work for a company that provided it a few years back. I'm just trying to figure out if you're actually being serious with your claim that everyone with a database backed website should be running it from isolated servers? If you actually understand the implications of this or if you're just making nonsensical off the cuff remarks? I wonder how many tens or hundreds of millions of servers that would add to the Internet.

Of course I'm being serious. Don't run serious applications from the shared MySQL databases at shared hosting providers. You seem shocked that I'm saying this, but we work with a lot of very young startups and I have never met one running their app off a Dreamhost-style shared server.

Tens or hundreds of millions of database servers? That's hyperbolic.

Do you really think that we've come close to eliminating all the vulnerabilities inside a MySQL session, post-authentication? Because what you're arguing is effectively that application owners should trust that MySQL is resilient against attackers who can get an authenticated handle to their own database and run nearly arbitrary SQL statements against it. You think all the code in the MySQL query parser, the planner, and the various storage backends have been fully audited? This is a project that didn't even get authentication right.

Right. By this logic MySQL doesn't need authentication, never mind privileges.

Honestly? In most real-world applications, MySQL authentication and privileges have absolutely no effect on security.

Again, you're completely ignoring shared hosting systems. You have a very strange understanding of what "real-world applications" means when it comes to MySQL.

One term for the applications you're thinking about is "certified pre-owned". Michel Zalewski just made a handy logo for those people to slap on the bottom of their front page:

http://lcamtuf.blogspot.com/2012/06/this-page-is-now-certifi...

"In most real-world applications, MySQL authentication and privileges have absolutely no effect on security."

To me, that makes as much sense as:

"In most real-world applications, file permissions have absolutely no effect on security."

Or:

"In most real-world applications, running services under isolated uids instead of running everything as root has absolutely no effect on security."

It's a bold claim, but one made with no evidence to back it up.

What is the best way to enable replication without access to the database? ssh tunnel?

Best? Probably an infrastructure VPN across databases. Have you database listen on the MPLS link and limit access to the service based on IP addresses and strong authentication.

Cheapest? SSH tunnel or openvpn point to point link.

Vpn, gre, whatever flavour you like. That's for replication between sites though. Internally there's nothing wrong with direct connection.

I agree with the sentiment, but just to be clear: There are valid reasons to allow MySQL to talk to the wider internet, the most important one being off-site replication.

If you do need your mysql to be exposed to the internet you can use bind_address=x.x.x.x or some kind of firewall.

metasploit module:

https://github.com/rapid7/metasploit-framework/blob/master/m...

With this logic, let's not secure any software that you cannot connect to it directly.

Before we forked LedgerSMB, the SQL-Ledger author's attitude was that unless you can exploit the software without logging in it doesn't count. It's only accounting software anyway so why would anyone want to break in? Not only that but timestamps were perfectly acceptable as session id's and they didn't even have to be stored on the server, just checked to see if they were recent.

Thus began years of efforts on our part of security fixes, which I would not have started except that I had customers to support.

What "logic" is this? Where are people getting the idea that I'm instructing them not to patch MySQL?

While your advice may be prudent, that does not excuse poor security.

Reply on news.ycombinator.com