When security goes right
daemonology.net
daemonology.net
"I don't see why this was a security problem in the first place. No personally identifiable data was disclosed. What does it matter if you can view anonymous traffic graphs from other customers?"
I hate it when people say this sort of thing. That just indicates they can't see a potential exploit, not that it won't be one potential aspect of an attack. Honestly, attackers - regardless of their morality - will tend to look at things from a viewpoint others haven't imagined. It's best to give them as few avenues as possible.
You're leaking information that you aren't supposed to.
There is no benefit or legitimate use case for this leak; it is a leak.
You gotta plug it.
Elaborate, sure, but plausible. The more important part of the leak is all the account numbers anyway :).
I'd be surprised if this constant chatter isn't visible on the graphs.
That means: The attacker can track rather well whether the person is at home or not. Or at least make informed guesses. They plan a robbery during a holiday. Or, much more malicious: some attack on the persons other web properties like their iCloud and Amazon account, as presented a while ago.[1]
Case B: Consider that the attacker doesn't have the account number. Another leak in their platform allows them to link personal information with account numbers. In itself, that info is also not very interesting and would also not be "dangerous".
Because the leak of the traffic graphs exists, they go on and plan the biggest heist in Toronto ever :).
A good exploit is often the combination of multiple seemingly innocuous flaws. [1] is also a good example of that.
[1]: http://www.emptyage.com/post/28679875595/yes-i-was-hacked-ha...
Also how many customers on which products - some graphs will top out at 4Mb and some at 25Mb, etc.
And it implies there wasn't a thorough security review on that server-which-has-access-to-customer-data, so any malicious attacker might turn more attention to it for a way into more access.
Cperciva says the access required a valid login, it's plausible that login could be customer number + password, which would make brute forcing easier with a list of current customer numbers.
Hmm
If they're big enough to have customers across Canada, usage patterns could reveal timezones. And whether the residence is unoccupied during work hours / what typical work hours are.
Longer term usage patterns could reveal if they tend to go out at nights, on weekends, over holiday seasons.
What else... maybe usage drops correlate with major sports events for some profiling? Maybe
Also how many customers on which products - some graphs will top out at 4Mb and some at 25Mb, etc.
And it implies there wasn't a thorough security review on that server-which-has-access-to-customer-data, so any malicious attacker might turn more attention to it for a way into more access.
Cperciva says the access required a valid login, it's plausible that login could be customer number + password, which would make brute forcing easier with a list of current customer numbers.
Hmm
If they're big enough to have customers across Canada, usage patterns could reveal timezones. And whether the residence is unoccupied during work hours / what typical work hours are.
Longer term usage patterns could reveal if they tend to go out at nights, on weekends, over holiday seasons.
What else... maybe usage drops correlate with major sports events for some profiling?
If you want to make it a 15 minute project, write a bit of customer-facing "We take your security seriously. That's why we encrypt all data with bank-grade security..." copy above or adjacent to the researcher-focused payload.
Good examples (I picked their disclosure pages rather than the security marketing pages) include:
https://basecamp.com/security/response https://www.twilio.com/docs/security/disclosure
It seems that even if I'm practicing full disclosure, telling the author at the same time I tell the world is a 1st order goal.
That is correct. If the world knows, then Scott will also be informed.
> It seems that even if I'm practicing full disclosure, telling the author at the same time I tell the world is a 1st order goal.
That is your choice, you are not obligated to do so with Scott.
If he makes a mistake, he wants hackers to call attention to it. He wants people to see his mistakes and how he responds. Maybe follow his example: to accept mistakes graciously, and immediately issue a patch that adequately addresses them.
If he gets burned in the process, that's the price Scott is willing to pay to improve.
Or maybe he's just cocky and is bluffing everyone because he thinks he's too good of a programmer to make a security-affecting mistake. Only way to find out is to audit his open source code and drop 0days onto Full Disclosure ;)
I understand wanting people to call attention to mistakes, and wanting that call-out to be clear and loud. I understand accepting that feedback.
But if it were me, my step #1 would be "disclose this via FD or whatever dispersal method you feel is appropriate, and tack my-address@example.org onto the CC line. That way we make sure I get your feedback and can address it". The goal of getting the author involved as part of step 1 isn't to hide from mistakes, it's to make sure the author doesn't get left in the dark just because they miss a mailing list digest line.
I intended to address this with my comment here:
> If he gets burned in the process, that's the price Scott is willing to pay to improve.
If Scott gets left in the dark, he feels that it is his fault for making a coding error in the first place. At this point, he no longer deserves to be enlightened. If the vulnerability discoverer feels like being nice and sharing this information first or simultaneously, wonderful. But if they botch it or maliciously post it everywhere else in the world, then no hard feelings. If public knowledge, eventually the problem will be fixed.
The key motive here is that at no point are third parties bound to regulate their behavior or self-censor. At no point will rudeness and/or publicly disemminating exploit code lead to any sort of criminal liability so long as the targets include Scott, Scott's code, and any systems solely under Scott's control.
Scott carries no legal stick. As a third-party security researcher with no business relationship with Scott, you should be empowered to give Scott as much advance/simultaneous notice as you feel is appropriate. With no requirements.
Let me frame it another way: The very act of publishing a security vulnerability benefits two parties: The publisher/vendor/author of the code that contains the vulnerability, and the public. In the case of Scott's open source software, the interest that matters most is the public interest. The public should be informed so they can decide whether or not they wish to continue to trust the code quality that Scott produces.
So what if Scott's servers get rooted and rm'd? He'll wipe them and write better code next time.
At no point will Scott impose any restriction on what you decide to do with your ideas that were inspired by reading his work. Even if your mind goes to dark places. All he asks is, just don't hurt the public. He's not exactly in a position to waive the right for the general public to press charges if you hack into their systems.
experienced reporters will recognize that the flaw they are reporting is already publicly visible and the thing that is the most important is getting attention and fixing the issue, secrecy won't help. you will get garbage reported to you encrypted, stuff like "I can see that your website uses PHP because there are URLS that end in .php".
If a company uses whois privacy I don't do business with them as a rule.
I don't really understand the way ICANN currently rhymes their 'whois data should be accurate' policy with the 'we allow the use of anonymization services' exception on that policy.
In the UK whois privacy is only for non-trading individuals (i.e. for personal Web sites) and always has been I think.
Then again, it's totally up to ccTLD operators to decide on what details are published in WHOIS. gTLDs (those longer then two letters long) have to stick to ICANN's policies.
Now, what is the case is that there's been a crackdown, due to pressure from law enforcement agencies, on incorrect details being set on domains. However, this actually has no effect on what's published in WHOIS because the only actual requirement is that the registrar have the correct details for the domain on record and escrowed. So long as the registrar has correct details, the registrar can mask the detail in WHOIS however is applicable, so long as it's clear that WHOIS privacy is in place.
There's hiding and then there's hiding.
If the number leads to a phone center the business contracted with and there's no real way out of that to the people who make decisions, is that not a form of hiding?
Except the overwhelming majority of customers has no idea what 'whois' even is and couldn't care less what info yours may or may not contain...
The only people who do care are spammers and scammers.
Whois-databases are a goldmine for them, and that's why whois-anonymization is a Very Good Idea™.
There are reasons to want privacy of course but not in the case of a business with a business address that most likely (say the local cake shop?) already puts their address on their website.
As a business, why wouldn't you want your contact info to be public? It's another piece of marketing.
Even more absurd to me is people who own domains who clearly want to sell them (let's say they are listed on SEDO or Afternic etc.) and they have privacy on their whois record. I mean get a PO box if you don't want your home address and you don't have a business address. If your domain is so valuable or if you own many we are talking $100 per year approx for an address. Use a google voice number for the phone number.
Lastly, lack of public info on ownership (so no trail of ownership at whois history) makes it much harder to prove you own the domain if something happens at the registrar. You are depending on them to have all their records in order. If they get hacked, go out of business and so on you could have a problem proving ownership. (However small it's not worth the risk).
ICANN does require registrars to archive whois data (with Iron Mountain) however since we don't offer privacy I'm not sure whether they require the underlying ownership (if you want to call it that) to be archived. Also many of the early privacy programs actually put ownership in the registrar's hands but had a separate contract with the actual registrant (not sure if that is needed anymore).
Personally, I don't think WHOIS privacy is something a business ought to be using, but it's perfectly legitimate for private individuals to use it.
> The large registrars push this as a profit center and/or some kind of benefit to enhance their offerings.
It is, because it's something the registrar can do at little or no cost. Given registrars work at thin margins, there are good reasons why even smaller registrars tend to offer WHOIS privacy.
> Lastly, lack of public info on ownership (so no trail of ownership at whois history) makes it much harder to prove you own the domain if something happens at the registrar. You are depending on them to have all their records in order. If they get hacked, go out of business and so on you could have a problem proving ownership. (However small it's not worth the risk).
That's precisely what data escrow through Iron Mountain is intended for, and why ICANN have been enforcing RDE since the failure of RegisterFly.
The key thing to keep in mind is that what is shown in WHOIS isn't necessarily what needs to be escrowed. A registrant is meant to provide accurate data to the registrar, and it's this data that has to be escrowed, but this data doesn't have to be what's published in WHOIS. This is why the only way to legitimately do WHOIS privacy is through the registrar of record. Anybody who's dumb enough to use a third party is asking for trouble.
If you're not escrowing the underlying ownership information (that is, the real registrant, admin, tech and billing contact details), then you're in breach of the RAA.
> Also many of the early privacy programs actually put ownership in the registrar's hands but had a separate contract with the actual registrant (not sure if that is needed anymore).
That can be solved by making it clear in the WHOIS details that the details displayed in WHOIS are for an agent acting the actual registrant. The registrar I work for also forwards any details sent to the email address published in WHOIS with WHOIS privacy on to the actual contact behind the scenes. Also, the registrar is required to escrow the real details.
Dropbox uses the same number for all three.
Neither company uses whois privacy. Agreed that for plenty of companies you'd end up in an IVR system but that seems to go for every kind of company these days, not just internet companies.
While a company should definitely have a whois page, most startups make it abundantly clear how to get ahold of someone at the company with large "Contact Us" buttons.
I've never contacted a startup to report a security issue, gotten a response (some don't respond), and had it not be from someone in a position to have the issue worked on immediately.
Hats off to Colin for the (brave) good deed.
If you're going to test, do it anonymously.
If you find anything, don't report it.
Anonymity and good OpSec are the only things that can guarantee to keep American hackers out of prison.
I'm not familiar enough with US law to know if a prosecution could be possible there, but regardless of the law might say I think juries can be influenced by the clear intentions of the accused.
I was about to see what happened if I changed the number, when I realised that I am a foreign citizen in a country with notoriously unpleasantly drafted laws about computer security.
And then I stopped. It's almost certain that nothing would go awry, but the mere act of gently poking might be interpreted in an unfavourable light. Losing a work visa is very easy and I very much don't want to.
If you're accessing a system a certain way (i.e. accessing certain parts of an application), then your traffic abruptly stops and a Tor exit node IP picks back up, you're hosed.
Tor may not be enough. To really be safe, use the precautions in http://pastebin.com/cRYvK4jb (this guy hacked finfisher and got away with it, so they know what they're doing.) Mostly whonix, truecrypt, and tor.
(Come to think of it, why hasn't that post been taken down? Is there nothing against pastebin's tos in there? They took down all the sony leaks.)
I would never go back and forth on the same page between tor and clearnet. I assumed that level of caution was obvious, especially to a fellow hacker (we're talking about vulnerability-noticers here, after all.)
Indeed, in a sane country where a judge would most likely would throw it out in pretrial you could spend tens of thousands of dollars getting a jury to in a less sane country.
Possibly true, but US juries are explicitly instructed (ad nauseum) not to do this and potential jurors are often removed from the pool if they seem unlikely to follow through. So, you know, be careful. (addressed less to cperciva than to other readers)
I wonder if instead one could use mod_rewrite (assuming Apache), and check the URL against %{REMOTE_USER}.
Basically, it's a poor man's capability system: once one has a URL to one's graphs, one can easily share them around, but if one doesn't have the URL and doesn't have the secret, one cannot determine the URL, thanks to the power of a decent hash function in the HMAC.
That is a really interesting reason to go with the popular open source solution. I guess I don't always follow that advice, but I wonder why I didn't think of it as a security decision.
My biggest gripe with Novus (besides limited coverage) is they have no unlimited bandwidth offering like my current provider has. Despite that, I'm still considering switching to them for those awesome upload speeds. In light of this, I'm probably going to sign up.
Last I heard, when you went over your bandwidth limit with Novus they cut you off to prevent overages. I really liked this because you have the freedom to call them and confirm that you're fine with additional charges and they'll immediately reconnect you, but I'm curious about if they still do this. Any current customers able to clarify?
Personally I've used about 20 GB in the past week, so I'm not worried about hitting my 500 GB cap.
Novus doesn't appear to offer that.
So personally, I'd switch to Teksavvy instead, and I will if I ever have any problems with Telus (but so far I haven't).
(Happy TekSavvy customer intentionally paying more than Novus would cost because they were doing nice things politically, and the 25Mb plan is ~$5 difference. Might switch if I get a faster plan, though.)
Plus they had Georges Laraque in their ads.
There's a need for extremely private registrations that still have a feedback channel. A guy I know has such a registrar, but I doubt there's any means of contacting any site's operator for technical, legal or other matters. (The whois postal address always lists somewhere Europe, but there's absolutely no published details. So it's a registrar that's about as private as allowable.)
Out of curiosity, how long do you think it would take to where you wouldn't be concerned? It seems hard to me to be really concerned without knowing exactly what changes were made, how many people were involved, and what kind of process they have in place for pushing code out the door.
I can only speak for myself, but I never push any non-critical changes out on a customer facing application in less than a day. I like to think about what I have done overnight just to make sure I am not breaking something. Even doing this I have still managed to break things on occasion :)
Use a cron job to delete all the generated graphs every minute.
I guess if it's just a stopgap until a better solution could be implemented then it's fine.
If I'm understanding correctly, I think this problem can be elegantly solved with Macaroons: http://research.google.com/pubs/pub41892.html
And it's relatively common issue with auth on the web. I think Facebook has (or had) this problem too. You can generally right click and "copy link" to get a .jpg URL, and send around people's pictures without any auth.
Basically the problem is when there are two web servers, a "dynamic" one with auth, and a static one that serves images. The static one is often a CDN.
Macaroons are basically a simple technique for decentralized auth, involving HMAC chaining. In this setting, the static server would first give the dynamic server a macaroon M authorizing ALL pictures.
At serving time, the dynamic server authorizes the user for a particular request. That request will have an <img src="acct123.jpg"> link. The dynamic server will take the macaroon M, and add a CAVEAT that the file must be "acct123.jpg", yielding Macaroon M2.
The client gets the restricted macaroon M2 with the HTML, and sends it back to the static server to retrieve the .jpg. The server can 1) verify that M2 is derived from the original M, and 2) read the caveat from the dynamic server, proving that the user was authorized for the image acct123.jpg (and only that image). The HMAC chain is constructed so that the client can't remove the caveat and get access to all pictures.
Basically what happened is that the static server DELEGATED auth logic for its resources to the dynamic server. In figure 2 of the paper, the static server would be TS (target service), and the dynamic server is IS (intermediate service).
The static server still needs extra code for Macaroons, which existing CDNs and static servers don't currently have. It would be cool to have an Nginx plugin that does this. But the key point is that it is preserving the original intention behind the static/dynamic split: performance.
In less performance sensitive context, you would have a web server perform custom auth logic, and then just read() the static file from disk and serve it over HTTP. This is likely to be many times slower than say Nginx. With the Macaroons, you can authorize ONCE in the dynamic server, and then PROVE to the static server that the auth decision was made. So all the .jpg requests can be fast and only hit the static server. The HMAC calculations are just hashing so they are cheap. It is symmetric crypto, with the shared HMAC secret.
The paper has some other use cases and is definitely worth a read. I'm thinking about using this technique for a project. I'm interested in opinions from crypto/security folks.
The HMAC verification is just doing MD5/SHA1 hashes and comparing for equality, which all CPU (and really CPU, not CPU + memory, since the data is so small). It's tens of microseconds of CPU implemented in JavaScript (Table II). I'm sure it will be single digit microseconds or less implemented in C, so that's at least 100K - 1M req/s. The CPU will be negligible compared to the rest of the workload.
The performance issues with naive solutions for auth of static files are pretty different: hitting databases for auth checks, copying data through two process, context switches, etc. Those are things likely to slow down a static file serving workload.
There is some extra "complexity", but I think it's almost the simplest solution you can think of for auth, even ignoring performance. It's a lot simpler and more robust than say putting a database in the request path.
The implementation isn't as big as it may appear. There is an open source library that is less than 2K lines of C:
https://github.com/rescrv/libmacaroons/blob/master/macaroons...
What I described is probably 200 lines of C or less. Again you are just verifying an HMAC chain. And the code for hash functions is stuff that probably already appears in all web servers anyway. It's using only the simpler "first party caveats" and not the "third party caveats".
The only problem I see with such approach is lack of expiration policy (other than re-loading the file under a different name every now and again).
The idea is to limit the power of the credential, so that when it's stolen, the negative impact is mitigated.
In the paper, they compare macaroons to plain cookies. When you steal a cookie these days, you can take control of an entire account -- maybe even change the password, etc.
In this situation, with the simple HMAC / guid / hash, once somebody "steals" the URL, then it can be leaked to everybody, forever. It can't be fixed without "breaking" the app.
The macaroon technique can also solve the expiration issue. In my post, I described how you would add a caveat for the filename. You can also add a caveat based on time. So then stealing the Macaroon would only authorize the attacker for a limited team -- even a single request (this is talked about in the paper).
A legitimate user can be minted new macaroons constantly based on his login. But the attacker can be locked out based on the expiration. So the attacker's problem is elevated from stealing a single cookie/macaroon to stealing either the login or ALL macaroons, which is generally harder. With Macaroons, you don't have credentials of full power constantly traveling back and forth over the network.
That is my understanding anyway... actually implementing it will probably reveal some more insights.