Gone in Six Characters: Short URLs Considered Harmful for Cloud Services
freedom-to-tinker.com
freedom-to-tinker.com
- OneDrive "[...] reiterated that the issues we discovered do not qualify as a security vulnerability"
- Google Maps "[...] responded immediately. All newly generated goo.gl/maps URLs have 11- or 12-character tokens, and Google deployed defenses to limit the scanning of the existing URLs."
Well done, Microsoft!
I'm considering migrating away from Microsoft products because of this; they offer bounties for important bugs but the way they handle reports is horrible.
True, none of those were security issues.
I suggest you take the long view and compare how Microsoft handled security disclosures in the past ("That vulnerability is entirely theoretical.") compared with today (inviting hackers to their oncampus Bluehat conference, sponsoring CanSecWest, etc). Things could always get better, but they've come a long way.
More specifically, "only" 2 weeks to issue a security fix is actually pretty good for thick client/desktop software. It's less than ideal for something like a web app where they control all the machines that need to adopt the fix, but still. Also, the severity of reported issue is a factor in when something gets fixed.
Consider looking at something like rfp's RFPolicy if you'd like guidance on how to disclose in a reasonable, timely way
I was only voicing my personal experience, which has been very poor (maybe the team, or the seriousness of the bugs), but in general I have heard some good things, especially for truly critical issues.
When did Facebook become the pinnacle of security response? The last thing I read about them was pretty horrible. Much worse than the Microsoft response here.
That's actually even more of a reason to migrate away from Microsoft.
it also says existing urls are still vulnerable
For mixed-case and digits, one requires at least 22 truly-random characters to provide 128 bits of security (43 for 256-bit security).
Exploring it with anti-brute force detection from the keeper of said URLs? Good luck. They could easily limit any ip to 10 URLs per hour or something and make it impossible to scan.
Then an attacker would just use a botnet. Granted, he can probably get interesting items off of the computers in his botnet too.
Not easy, but not impossible. And it's not the case that an attacker is looking for one interesting URL: he's looking for any interesting URLs. Depending on the number of URLs stored in the service, and the fraction which are at all interesting, it may very well be worth the attacker's bother.
As an aside, why the heck is my post so heavily downvoted? It's factual: the given lengths are not long enough to be secure from brute-forcing; they probably will be brute-forced.
I must admit that I don't understand how the number of characters corresponds to bits of entropy. Know of a resource that explains this?
log(num_options)/log(2)
There are 26 letters; mixed-case doubles that for 52 choices; digits add 10 for a total of 62 possible choices for each character. That means that 2 characters have 62^2 possible configurations, 3 62^3 and so forth. If you take the resulting number and calculate the ceiling of its logarithm in base 2, you get the number of bits needed to represent it:
(ceiling (log (expt 62 11) 2)) → 66 -0.5038376In turn, that means the entire space is ~10^20 URLs.(smidgen less, 710^19)
Let's assume 1B users, with 1K URLs belonging to them on average. That's 10^12 URLs. Which means, on average, you query 10^8 URLs until you hit the first one.
Let's further assume you could actually query 10^5 URLs/s. That means a single* URL requires query rates for 20 minutes.
Sure, theoretically that's doable. Except you'd cause query rate and error rate to spike, and the setup to do so would be quite expensive.
So, in the best case, after those 20 minutes, you have a random picture of a dog, or a map.
Having a network capable of running 10K qps and risking detection to find, maybe, one picture every 20 minutes? There's just not the incentive to do that. There are many more interesting avenues for this.
And since it's bandwidth-bound, not CPU bound, that speed is not going to rapidly accelerate.
Edit: The issue would be with existing links, but it's difficult to change existing links for obvious reasons. Both MS And Google at this point have calculated that deleting existing vulnerable links would be worse than the security issues presented here.
There's two main concerns: the first being that you use a URL differently---and think of it differently---than a password; and the URL persists in various forms.
Accessing a resource on the Web might be logged in numerous places---e.g. your web history and internal network logs or MITM if not over https---and might be sent as metadata, like a referer header to another website. Imagine if all of your passwords were logged by your client any time you entered them in.
Given that, if a company is providing URL shortening for their product (such as One Drive), that should be their responsibility to protect the users.
I find it amazing that they didn't do this in advance and needed to have this happen to know to fix it.
I remember way back in 1996 or 1997 being able to do URL scans of UPS shipping tracking numbers. It was trivial to bring up the ship to address for any given shipper (for all of there customers) as long as you had at least one of a shippers tracking numbers (then just alter to suit). [1]
[1] So in other words if I received a UPS package with a tracking number for ABC company I could easily see every else that they were shipping to including names, addresses.
http://gizmodo.com/hacker-says-url-trick-grants-access-other...
Praising google and blaming microsoft on that is actually stupid and harmful — this "concerned about security" image is nothing but marketing move for naïve people. If shortened urls are twice as long as before, does it mean that now it's safe to have some confidential information assigned to them left without authentication? I bet it doesn't. But it sort of implies so. The url shortening service just got worse, by being twice less efficient in what it's supposed to do: shorten urls. But not more safe, not at all.
Let's see what happens when we apply this logic to cryptography...
If somebody can access you data just by guessing it's key — well, it it unprotected, so implicitly it is supposed to be accessed by anyone. No matter how long the key is.
And in fact your comment and the fact that you think these things are comparable is exactly what is dangerous about the situation, and not the damn short urls themselves. It's like some guys forcing drillmakers to start making drills out of rubber, because some idiot tried to pick his teeth with turned-on drill and got killed. See how dangerous drills are! Well, yeah, they are pretty dangerous, but how about just not picking your teeth with the fucking drill, and not applying "toothpicking logic to drillmaking"?
By the way, the guy with rubber drill still can kill himself with that — it's just less efficient for both drilling wood and killing idiots. Exactly like what it is with "longer shortened urls.
Even if you 'don't have anything to hide', you don't want anyone to know that you sent someone the directions to a planned parenthood center. Not because you think it's a bad thing to do, but because the publicity of this information could be harmful to you.
"If you've got nothing to protect..." definitely has a different feeling to it.
Next time someone presents the 'if you have nothing to hide argument', tell them to send you a video of each of their family members on the toilet.
We're not supposed to let it know everything we do or say, while it is increasingly more secretive with its own actions. It should be the other way around.
But most of us have bought into the government's propaganda that "if you have something to hide, then you must be a criminal", which isn't that different from the argument that "if you oppose the war in Iraq (or elsewhere) then you're not a patriot, or you're a terrorist sympathizer".
Stop buying the U.S. government's bullshit propaganda. Start with rejecting it by default, and then maybe consider whether it's right or wrong, not by accepting it by default as true. We've been burned too many times trusting the government's lies. We should've learned our lesson by now.
It's the 'none of your business' facet of privacy.
Even if people do manage to live 100% criminal free (which may not be possible, but even if it is), sharing information like addresses, account numbers, DNA, fingerprints, etc. makes one vulnerable.
Even more surprising is the number of people on here who don't understand why this is problematic, essentially blaming the victim for not understanding that their private channel is leaking information. It certainly is not obvious to the general public, and wasn't obvious to the people who implemented these services, that a side-effect of a shortener with insufficient entropy is leaking information from private channels.
Fundamentally, long-urls-as-security makes mathematical sense but doesn't actually work with the way the web behaves. URLs are shared constantly, freely.
If we had everybody using the same auth systems so we could do whitelists without "oh you need a google account" or "oh you need an MS account" then whitelists would be a solition, but In Real Life whitelists get in the way so instead people just rip out the security altogether and count on the security of the URL.
Imho, public-with-shared-password would be the right feature to add, even though it's anachronistic.
Driving directions by themselves are usually pretty innocuous. When one end is something like an abortion clinic, it gets a bit more sensitive; when the other end is a specific house, even more so. And I don't think you can expect driving directions to ever get password protected.
Some links are like words - merely a reference; some are like sentences - describing some relation or fact; and some are like passwords - describing how to find or access something that you wouldn't otherwise know.
The first are not revealing when out of context; the last are always dubious. It's the middle case, describing a relation (like driving directions) that's most problematic.
Unless you're sending your link over a relatively short fixed-length limited medium like Twitter or SMS, there is no fucking point.
I've seen people post links to download apps, which go from their own site > some random bit.ly/etc URL > dropbox. I'm already seriously doubting if I want to run your app if you can't manage something better than dropbox file sharing, but to then rely on a bit.ly URL that could go fucking anywhere, when you're just putting the link on a webpage is beyond belief.
And bit.ly gives you analytics. That's their service.
2. Long urls still sometimes break when handled by some email clients.
2) Analytics. I've seen many times different shortened urls used in different locations for the same campaign. I assume that makes it much easier to track your offline marketing channels.
Add a "+" to any bitly or goo.gl link and see what I mean. https://goo.gl/forms/9fA366pQ1f+
I use goo.gl to see how many people click a link I share on facebook too.
I also strongly suspect that a lot of them are being used more for analytics than URL shortening, which also annoys me to no end.
If your security depends on someone not guessing a URL, shortened or not, you're doing something wrong.
Why? Done properly, a secure URL should be as long and contain as much entropy as a strong password.
Are all systems which depend on someone not guessing your strong password wrong? Are practically all encryption schemes wrong?
I'm not saying that it's necessarily wrong to use a long URL as security; sometimes the problem constraint means that is the only way to do it. I've done it on rare occasion. But I also made sure that management knew there was a risk here.
The latter is up to server design, the former is an interesting point.
Browsers are unlikely to cache get parameters, but these things might end up in history etc..
The path is exactly as secure as authorization headers. Network logs will not show the path of SSL requests (it's encrypted).
If you give your link to a dodgy search engine, you've lost.
No, but if your security system has a password and only one account shared by everyone where the password is shared as plain text (typically just emailed from person to person) without any sort of user access logging then it's got major problems.
With just a URL you have no idea who has access and the only way of revoking access to anyone is to permanently move the resource and then get back in contact with everyone who should have access by sending the new "password" around.
> Are practically all encryption schemes wrong?
An encryption scheme that involves the users sharing the only password through far less secure channels?
This is to say that the whole url-shortening business should still be improved, even though you're probably not going to use it for your private stuff where you want access control.
I should be able to just say "here's the link, the password is burp441toaster, no spaces all lowercase".
Craigslist does this and it's a great system for a Craigslist post. You can sign up for an account if you want but you can also post without an account and you get a unique url as a password to edit or delete your post. Craigslist posts are only good for 30 days and someone deleting your Craigslist post isn't the end of the world.
It's very possible to use urls as a password securely, password reset emails do it all the time.
Not as catchy, I admit.
Well, fast forward about two years, and that script is still sitting around on a forlorn webserver of mine. Somehow, I have no idea why, some random person ended up tweeting a link to it and it spread around a bit until the software vendor got wind of it. They ended up sending me a probably too-polite email asking if I could do something about it, and after a bit of back-and-forth I got instructions from them on how to enable more secure "long URLs" in the software (an option that I think was new since I made it, so I wonder if I may have actually inspired it...) and added those to the bottom of the page.
It's long gone now, and to be honest I can't remember which app was affected. Possibly tinygrab.
The point of this anecdote is that the problem is not at all new, and the problem of how to deal with it isn't new either. I suggested to the developer at the time that they should probably use long URLs by default, but it seems users just like those short URLs too much. Going to non-sequential assignment would have helped, but the space was still just too small.
Really, I think the fix is just communication. Microsoft's workflow used to be sensible in that OneDrive gave you a long URL and then you had to click another button to get a short one. That second click should come with a warning that there should be no sensitive information in the document, and it will potentially become public after shortening. Users will have to be trusted with the judgment, at least you've CYAd.
What is it with these large companies ignoring serious security issues while paying attention to smaller ones? I reported something to Facebook that was a moderate privacy concern and got a bug bounty. A few months later, I discovered that I could make Facebook falsely report the domain that a posted URL goes to, and they denied that it was even a bug. So I could share a URL on mydomain.com, customize the contents of the share posting ("Obama says he's going to nuke Russia"), and Facebook would show users in the post that the link goes to Whitehouse.gov or CNN.com or any other domain I choose. This still works perfectly.
These companies really need to take a look at the analytical abilities of those they are employing to screen bug reports.
"Brian" probably looked into it, knowing that obscurity != security, but got a response back from the group responsible that was the way they intended it to work, and that group's management wasn't going to do anything about it. "Brian" may have even put messages in the right ear up the management chain such that it would actually effect the outcome.
The fact that the email exchange lasted 2 months before "Brian" said "Sorry, not a case." probably means that "Brian" was trying to make it happen and had actually done an analysis.
* Note: I'm NOT Brian. I've never worked for Microsoft.
> TL;DR: short URLs produced by bit.ly, goo.gl, and similar services are so short that they can be scanned by brute force.
This is not the issue for OneDrive. Everyone knew this already, right?
For Google Maps, it's definitely more nuanced. I'm glad Google acted swiftly.
> Our scan discovered a large number of Microsoft OneDrive accounts with private documents. Many of these accounts are unlocked and allow anyone to inject malware that will be automatically downloaded to users’ devices.
This is the issue for OneDrive. I'm not a OneDrive user, but if the documents are publicly editable per a setting the user controls, this isn't a "vulnerability" either.
These services advertise it as "editable by anybody with the link" not "editable by anybody"
There is an implication that people can't get the link without you giving it to them.
OneDrive generates short URLs for documents and folders using [..] the same tokens as bit.ly. [..] In our sample scan of 100,000,000 bit.ly URLs with randomly chosen 6-character tokens [..] 19,524 URLs lead to OneDrive/SkyDrive files and folders, most of them live. [..] From the URL to a single shared document (“seed”), one can construct the root URL [from which] it is easy to automatically discover URLs of other shared files and folders in the account
In other words, the links provided by these shorteners contain authorization information. And:
Around 7% of the OneDrive folders discovered in this fashion allow writing.
And in the case of Google Maps:
goo.gl/maps URLs used 5-character tokens. Our sample random scan of these URLs yielded 23,965,718 live links, of which 10% were for maps with driving directions. These include directions to and from many sensitive locations: clinics for specific diseases (including cancer and mental diseases), addiction treatment centers, abortion providers, correctional and juvenile detention facilities, payday and car-title lenders, gentlemen’s clubs, etc. The endpoints of driving directions often contain enough information (e.g., addresses of single-family residences) to uniquely identify the individuals who requested the directions
A better analogy would be when routers ship with a default world-viewable admin UI and admin as its password.
Do Google et al not have any kind of rate limiting which looks at suspicious behaviour like scanning lots of short URLs?
I think your expectations about the "smartness" of the public are not justified. It's not actually about smartness; it's about information theory. Not everybody is as up to date as you are.
Almost all tech can be used in the wrong way, this does not make the tech bad if use correctly.
URLs aren't secure, and shouldn't really be considered so.
No, it really isn't security. But yes, it really does happen, and probably a lot more than you'd think.
Even if URL shortening is a feature that users are aware exists, the consequences of it certainly aren't immediately clear, and to my knowledge, not many of these services include a way to disable the generation of a shortened link or have a means to prevent this type of scanning from happening.
Interestingly, a lot of email client also replace links by their own creating similar risks, but nobody talks about those...
I have to applaud Slack for displaying and linking to links the they were meant to.
{"error":{"code":"generalException","message":"General Exception While Processing"}}
[0] - at the cost of usability
Otherwise you end up with people just sending around "go to this url with this login and password" through email and chat instead, which is slightly harder to automatically exploit but not really more secure.
But I don't think you can find out one's imgur username or more of their photos from just having a link to one of their photos.
I also tested from an arbitrary image in an album, trying to go backwards to find out what album it belongs to, but that doesn't seem possible either.
Even now, if you connect to your favorite “trusted” long domain names, nothing stops that from being totally hijacked by an untrustworthy Internet service provider or other entity that has access (or more insidiously, inserting crap like ads that were not in the original source).
And heck, long URLs are suspicious as well. I’m sure by now everyone has received one of those ridiculous "important@facebook.com.kdsjfksdjfkdsfjdskfjdskfjdskfjdskfjdskfjdskfjdskfs.oopsmalware.com".
The push should be for broader adoption of mechanisms that make it hard to subvert what you download, and easy to verify what will happen when you click a link.
The part about automatically-generated short URLs in MS docs is (was?) worriesome. Few users understand the implication of having a public URL that directly references their document which, in all likelihood, were intended for a restricted audience.
I should point out that Slack has the same bug, though their URLs are simply obfuscated with a longer token. Shameless plug: Cisco Spark[0] has solved that with end-to-end encryption.
[0] https://support.ciscospark.com/customer/en/portal/articles/1...
Giving my collaborators enough power to inject executables is far beyond my needs and my intent when I make the doc "open". At worst I'd expect to find a document edited with a link to an external malware exe, not some horrifying autorun problem.
You could also do warnings when the user clicks a link ina publicly editable doc "this document is publicly editable, which means that any rando on the internet might have set this link, not just your buddy who made the doc. Are you really sure you want to go there?"