Hacker News Reader

Comment by jedberg | Hacker News Reader

I have only given this a moment's thought, but why not just publish the URL map as a text file or SQLLite DB? So at least we know where they went? I don't think it would be a privacy issue since the links are all public?

View on HN

It will include many URLs that are semi-private, like Google Docs that are shared via link.

If some URL is accessible via the open web, without authentication, then it is not really private.

What do you mean by accessible without authentication? My server will serve example.com/64-byte-random-code if you request it, but if you don’t know the code, I won’t serve it.

Obfuscation may hint that it's intended to be private, but it's certainly not authentication. And the keyspace for these goog.le short URL's are much smaller than a 64byte alphanumeric code.

Sure, but you have to make executive decisions on the behalf of people who aren't experts.

Making bad actors brute force the key space to find unlisted URLs could be a better scenario for most people.

People also upload unlisted Youtube videos and cloud docs so that they can easily share them with family. It doesn't mean you might as well share content that they thought was private.

I'm not seeing why there's a clear line where GET cannot be authentication but POST can.

Because there isn't a line? You can require auth for any of those HTTP methods. Or not require auth for any of them.

I mean, going by that argument a username + password is also just obfuscation. Generating a unique 64 byte code is even more secure than this, IF it's handled correctly.

That's not any better than what archiveteam is doing. They're brute forcing the URLs to capture all of them. So privacy won't really matter here.

Then use something like argon2 on the keys, so you have to spend a long time to brute force them all similar to how it is today.

So exclude them

How?

How will they know a short link to a random PDF on S3 is potentially sensitive info?

I meant Google docs of which they know share settings

I'd rather see it as a searchable database, which I would think is super cheap and no maintenance for Google, and avoids these privacy issues. You can input a known goo.gl and get it's real URL, but can't just list everything out.

And then output the search results as a 302 redirect and it would just be continuing the service.

Are they all public? Where can I see them?

You can brute force them. They don't have passwords. The point is the only "security" is knowing the short URL.

I don't think so, but you can find the indexed urls here https://www.google.com/search?q=site%3A"goo.gl" it's about 9,6 million links. And those are what got indexed, it should be way more out there

I'm surprised Google indexes these short links. I expected them to resolve them to their canonical URL and index that instead, which is what they usually do when multiple URLs point to the same resource.

archiveteam has the list at over 2billion urls with over a billion left to archive.

Reply on news.ycombinator.com