PostgreSQL 9.2.4, 9.1.9, 9.0.13 and 8.4.17 released
postgresql.org
postgresql.org
An oversight in commit e710b65c1c56ca7b91f662c63d37ff2e72862a94 allowed
database names beginning with "-" to be treated as though they were secure
command-line switches; and this switch processing occurs before client
authentication, so that even an unprivileged remote attacker could exploit
the bug, needing only connectivity to the postmaster's port. Assorted
exploits for this are possible, some requiring a valid database login,
some not. The worst known problem is that the "-r" switch can be invoked
to redirect the process's stderr output, so that subsequent error messages
will be appended to any file the server can write. This can for example be
used to corrupt the server's configuration files, so that it will fail when
next restarted. Complete destruction of database tables is also possible.
Fix by keeping the database name extracted from a startup packet fully
separate from command-line switches, as had already been done with the
user name field.
The Postgres project thanks Mitsumasa Kondo for discovering this bug,
Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing
the full extent of the danger.
Security: CVE-2013-1899And to be fair: http://www.shodanhq.com/search?q=mysql
I suspect that's why they weren't able to tell people that if your db port is secure you're safe.
EDIT: No, the problem is in parsing, not the existing names: https://news.ycombinator.com/item?id=5492508
https://en.wikipedia.org/wiki/Tom_Lane_%28computer_scientist...
The Postgres project thanks Mitsumasa Kondo for discovering this bug, Kyotaro Horiguchi for drafting the fix, and Noah Misch for recognizing the full extent of the danger.
We can't exclude without further info that Tom contributed to the fix, just saying.
Knowing him after a few years spent following postgres dev process I bet on co-authoring at least :)
It seems that every time I search the archives regarding an issue I'm having: I find a reply authored by Tom Lane.
I believe all the heroku hosted postgresql servers are externally accessible and there's no way to filter access by IP.
Of course hindsight is always 20:20, but perhaps it's a good idea for heroku to consider adding some basic (optional) firewall layer to allow customers to control who can connect to the hosted db?
Disclaimer: I'm not a heroku customer. I did however consider moving our pg's over to them a little while ago.
I couldn't find a PG hosting provider with more useful features than Heroku - with that, it would be a killer option.
At the very least I think they should offer an option that only you to restrict access so only your dynos have access to the postgres port.
If only they could limit access to EC2 security groups, it would be amazing.
Security Groups work across accounts, so Heroku (or whoever) could let you provide your account ID and Security Group name, then authorise access from this group.
Another thing I was really hoping for but couldn't find with heroku, was being able to do a point-in-time-restores via the heroku web/cli interface. This would be a seriously nice feature if something like that was available...
It doesn't mean that other people like me wouldn't be interested in something like this if it existed.
The problem is the sheer number of Heroku Runtime machines which are located in a smattering of IP space, and rapidly and accurately propagating the firewall rules required for tight network access control as applications churn around in there...even then, there have been some reports of voluminous firewall rules causing obscure problems. Of course: the world is an obscure place and yet we can deal with it in time. Such a thing could be hardened I'm sure, but the amount of bookkeeping required is a bit terrifying, and experience suggests that will not go entirely smoothly or be easy to find bugs in. At the time this was reasoned out (maybe about two years ago?), it wasn't even widely known that Heroku offered any data storage service of significance. Early days.
So, the simple approach is to enable access from the entire Heroku Runtime layer. But who can put applications there? Anybody on the Internet. That's why the 'ingress' feature to poke a temporary hole in the database firewall was dropped as too marginal given the fairly severe inconvenience of it all -- it had the feel of a weird Heroku-ism, especially in light of the lack of attacks using unauthenticated clients on Postgres, until this date. In addition, what about all the other addons? There would need to be an API, and because of the nature of what it is doing (poking holes before beginning, say, TLS negotiation which requires even more round trips and is slow enough as it is) it would need to be stable, fast, accurate, and all that other good stuff, lest all addons effectively be rendered offline all at once.
Other more application-level approaches are possible (like tunneling all connections to a local unix socket, or something), but that's but a little strange, because it requires code injection of strange stuff into the running container, makes your URLs look funny, and so on. This model has been experimented with by some of my colleagues and staff of other, similar firms, and definitely has its attractive sides. Nevertheless, one of the general guidelines in our implementation choices is to not be too weird for someone moving on or off the service. These approaches are also not in and of themselves immune to DoS or security problems...they need careful auditing. Maybe another look is indicated. And again: what about every other addon? What about your local computer? Are you going to install some weird agent, open source or no, from every such addon?
My personal favorite pie in the sky option right now would be cordon off a slice of contiguous, publicly addressable (but not publicly accessible) IP space so that firewall rules could remain compact and slower changing, and also involve your local computer: imagine being able to VPN to two or three such networks simultaneously, because their addressing does not alias at all. But this is still in the realm of fantasy, and and probably would require looking at IPv6 to be able to segment the address space in a sane manner, which compounds another layer of stuff that can be buggy, even in such mundane details as address parsing.
So....there's some rambling, but I thought it perhaps useful to talk about some of the challenges here to motivate the discussion.
But seriously, I'm glad that I was not totally off base in letting my mind drift to IPv6. Thanks.
They took their repositories private to secretly develop the bug fix. Then they released the fixed versions along with what seem to be enough details to trigger the bug for anyone who hasn't patched.
Sure the patch contains the same information in source form, but if they'd gone light on details while saying "seriously, go get this", there'd probably be fewer curious vandals trying to delete your database while you're reading HN.
"Apply this patch, don't worry what it does, just do it" is not something I want to hear from my database vendor :-)
Had the repos remained public, this detailed information would have been available to a lot more people, a lot sooner. Temporarily "going dark" to work on the patch seems like an acceptable compromise.
Hiding the information just weakens the defender position, not the attacker position. Secrecy in implementation is not security, it is just stupidity.
They are open source, though, and many people who use it build from source. It is very very easy for complete amateurs to look through the source and see what changed in a manner of minutes.
Is there an attack vector if you run PostgreSQL locally, no untrusted users are able to create connection strings and do not allow remote access?
It seems to be no but I prefer to be sure ;)
Who is most at risk:
"Any system that allows unrestricted access to the PostgreSQL network port, such as users running PostgreSQL on a public cloud, is especially vulnerable. Users whose servers are only accessible on protected internal networks, or who have effective firewalling or other network access restrictions, are less vulnerable."
So looks like it's low risk but they're not willing to say no risk.
That means anything that gives local shell as any user that run normal tools, but potentially also a lot of other things.
E.g. any software that can be tricked to try to connect to a local address/port pair and send a suitable string.
That dramatically escalates any minor little hole that might otherwise not be a risk for you.
(That's a reminder to always verify before trusting any hostname/IP a user passes you that it's not a local address or address you have privileged access to, and to also consider internally firewalling connections between your various hosts down to just what you need)
The upgrade (on Debian based systems at least) is for libpq5, postgresql-9.1 and postgresql-client-9.1 . You shouldn't need to do anything else unless, for some strange reason, you actually do have a database starting with "-".
Also, I wish you a fine hike!
(probably commented too late, but I hope you had a nice hike, nonetheless)
> How can users protect themselves?
> * Download the update release and update all of your servers as soon as possible.
> * Ensure that PostgreSQL is not open to connections from untrusted networks.
> * Audit your database users to be certain that all logins require proper credentials, and that the only logins which exist are legitimate and in current use.
EDIT: added a quote from the FAQ for clarity.Because some folks need to go through a long, drawn out process to upgrade internal software and might already have some place in the pipeline to put the upgrade so it will get tested with everything else.
If its a security issue for your current installation, you go now and hope, but if it can wait for a release then you do that.
It's still important to apply the udates, as I wrote elsewhere, because it's one more way someone can mess you up badly if they have managed to penetrate other parts of our systems, but upgrades no matter how seemingly safe can also cause problems (fore example I test upgraded one VM that refused to come back up again afterwards - not Postgres' fault, but a config change someone had clearly not tested properly).
But in this case, knowing we have blocked the main attack vector means we can take the time to take a copy of each database VM we run, test apply the upgrade and verify everything works correctly before applying it live, instead of rushing it out.
The reason I wanted to not upgrade directly is that access was hard at the moment of posting. Upgrading should never wait too long but this was (luckily) defcon 1 for us.
Lots of folks complained about that (unintended) ActiveRecord change in Rails during the last security update.
PQconnectdb("host=127.0.0.1 dbname=-exploit user=postgres password=postgres port=5432");
EDIT: They are not overwritten but just appended to.
The vulnerability allows users to use a command-line switch for a PostgreSQL connection intended for single-user recovery mode while PostgreSQL is running in normal, multiuser mode. This can be used to harm the server.
> Fix insecure parsing of server command-line switches (Mitsumasa Kondo, Kyotaro Horiguchi)
So I assume command-line switch parsing is somehow involved in parsing the connection string (probably because the same connection strings can be used from API and from CLI?), I guess a database name with a leading `-` can be interpreted as a switch and execute corrupting commands.
edit: according to the dedicated FAQ:
> The vulnerability allows users to use a command-line switch for a PostgreSQL connection intended for single-user recovery mode while PostgreSQL is running in normal, multiuser mode. This can be used to harm the server.
PostgreSQL major versions are represented by the first two digit groups of the version number, e.g., 8.4. PostgreSQL minor versions are represented by the third group of version digits, e.g., 8.4.2 is the second minor release of 8.4. Minor releases never change the internal storage format and are always compatible with earlier and later minor releases of the same major version number, e.g., 8.4.2 is compatible with 8.4, 8.4.1 and 8.4.6. To update between compatible versions, you simply replace the executables while the server is down and restart the server. The data directory remains unchanged — minor upgrades are that simple.
> A connection request containing a database name that begins with "-" may be crafted to damage or destroy files within a server's data directory
I just. No words.
However this does not prevent any of your employees or other users of systems with access to use spipped from committing this attack. You still need a client somewhere and the server is still vulnerable.
Allowing remote connections from any IP to your database, like heroku apparently does, sounds kind of crazy to me. I can't believe they do it. But limiting and encrypting that access just limits, and does not eliminate your vulnerability to this bug.
---
Just to be really clear: Say your corporate blog stores it’s data in your main Postgres instance. As blogging engines tend to, it has a bug, and hackers succeed in using that to get access to your blog’s server. Even if you are using spiped to connect the 2 boxes they still have the ability to mess with your main database, on some other, probably much better secured, box. This bug is ugly.
Just because the attack vector looks simple doesn't mean the bug was obvious.
1: http://thehackernews.com/2012/06/cve-2012-2122-serious-mysql...
From the FAQ:
> Persistent Denial of Service: an unauthenticated attacker may use this vulnerability to cause PostgreSQL error messages to be appended to targeted files in the PostgreSQL data directory on the server. Files corrupted in this way may cause the database server to crash, and to refuse to restart. The database server can be fixed either by editing the files and removing the garbage text, or restoring from backup.
1) PostgreSQL does generally report which file was corrupted.
2) The PostgreSQL log output rarely looks similar to regular data, so it should be obvious to anyone looking what is wrong if they do look at the contents of the file.
And most importantly they can always contact a PostgreSQL expert who could repair it.
Very smart and conscientious people can mess things up here. Shaming serves no purpose.