Hacker News Reader

Comment by whughes | Hacker News Reader

Extra cloaking for malicious links? Bit.ly can't exactly check the original site to see if it's OK if it's hidden behind a funky redirection layer. I think that bit.ly was reasonable to have an interstitial, although maybe they should be more open about why and how to remove it.

View on HN

It's a standard http 301 redirect, not a "funky redirection layer".

    /Users/sjs % curl -i http://xrl.in/33qj
    HTTP/1.1 301 Permanent Redirect
    Date: Sun, 09 May 2010 00:38:19 GMT
    Server: Apache
    Location: http://www.donationcoder.com/CodingSnacks/index.php
    Content-Length: 0
    Content-Type: text/html

Right, but the problem is this scenario:

Bit.ly see's your xrl.in and does a request. They find 301 and the location at donationcoder.com. They conclude "this site is ok". Later, the xrl.in url is changed to <malware link>.

They aren't going to do a request to every url they're linking to on every click, obviously. So they'd only get the one chance.

Now, I'm not actually sure that xrl.in lets you change links after shortening. The point is that bit.ly doesn't know either.

Bit.ly could check a link to no_malware_here.com, which thereafter adds a 301 that redirects to your_computer_now_has_aids.com.

I think it is misleading to display that message based on the possibility of a redirection. Any page can do that, not just xrl.in.

I think that's the point. They check for a redirection and show an interstitial. They've white-listed a few other shortener services (the ones they know are redirecting to actual sites and presumably do their own redirection checks on)... but as far as I understand it any url that you try to shorten that does an immediate 301 gets the interstitial first.

If they let urls with redirects on them they can inadvertently bit.ly link directly to a malware site. They don't want to do that at all and take measures to prevent it. Checking urls against malware lists and not allowing redirects are just a couple, I'm sure there are more.

As for being misleading in the interstitial itself... I don't think so. They've updated it to be more clear about the issues with this link:

* Some URL-shorteners re-use their links, so bit.ly can't guarantee the validity of this link.

* Some URL-shorteners allow their links to be edited, so bit.ly can't tell where this link will lead you.

* Spam and malware is very often propagated by exploiting these loopholes, neither of which bit.ly allows for.

Easy solution: before serving that warning page for shortened URLs, dynamically determine the endpoint. If it does point to a bad site, then serve the warning page. Otherwise, do the right thing.

That isn't so easy when there could be 15 redirects between bit.ly and the malware site.

I can submit a link to any URL in my control and swap it out for something else later. That can apply to every single link on the web. By that logic they should maintain a whitelist of not only redirection services such as shorteners, but for each and every domain.

I'm all for whitelisting and being paranoid, but it just doesn't make sense here and it seems a bit like they're trying to make the competition look bad. This should really be a blacklist instead.

I'm not entirely sure but I believe the idea is that any 301 will cause bit.ly to show the interstitial. So you're right, it does apply to any site on the internet. It's just that you mostly notice it on other shortener services that haven't been whitelisted (blacklist makes no sense because it would then be bit.ly's responsibility to know of and check every other shortener service out there).

They could check the endpoint URL by following the HTTP headers on 301/302 redirects.

Reply on news.ycombinator.com