Google Warns: bit.ly Links Are Unsafe | Hacker News Reader

Google Warns: bit.ly Links Are Unsafe

google.com

110 pointsantoniotajuelo63 comments

Any URL shortening site that doesn't have an active block list is likely linking to some unsafe sites. Singling out bit.ly in this instance is frankly unfair.

Guess who else Google warns is linking to unsafe sites:

https://www.google.com/safebrowsing/diagnostic?site=google.c...

Can somebody explain the part where it says "google.com has infected plus.google.com" and other websites in your link?

Someone with a Google Plus account embedded content (likely a block of JavaScript, iframe, or Flash object) from a bit.ly URL. That content was in turn deemed malicious.

goo.gl too, with nearly 6,300 links to Trojans, way more than on the bit.ly report: https://www.google.com/safebrowsing/diagnostic?site=goo.gl

Although for goo.gl "6292349 pages we tested on the site over the past 90 days, 826 page(s) resulted in malicious software" and for bit.ly "91856 pages we tested on the site over the past 90 days, 721 page(s) resulted in malicious software". The % of pages on goo.gl with malicious software is much lower.

http://unshort.me/

Never fear.Also could you do some enlightening of others? Some people use URL shorteners by default when they are absolutely unnecessary. Not to mention privacy policy of these services is often uncomfortable at best.

But what if we're dealing with a bitly within a bitly? A 'bitly-ception'? It's bitly's all the way down... Does unshort.me recursively check? (Oh, and what if the bitly goes to an unmarked, safe-looking site with a js redirect or etc.?)

I believe unshort.me "unshorts" it all the way. I have no tested it as I'm a lazy useless individual, but feel free to try it out.

Edit: OK, I just did a quick test, "unshorting" t.co link that hides a bit.ly link gives the final link.

Hey thanks for mentioning http://unshort.me

I created http://unshort.me and you are correct it follows all the way until the shortened url is resolved unless it is a circular link. It also stores the result in a database so it works faster once someone resolves the URL.

I also created fuseurl.com. It used to be a bitly but for many URLs but I shut it down because it started to contain a lot spams and viruses.

Well, then let me thank you for making the service work and keeping it clean.

longurl.org is awesome to check what is behind a shortened URL.

Has an API too: http://longurl.org/api

HTTP has an API for that too, just make a HEAD request to the shortened URL and see where the Location header sends you

Yeah, sadly doesn’t always work. Goo.gl and youtu.be sometimes send an http page consisting of

<!DOCTYPE html> <meta http-equiv="refresh" content="0; url=https://www.youtube.com/watch?v=opoDBF_b-fg&feature=youtu.be...

Which makes it far more complicated. Another occasion where it’s appropriate to say “Fuck you, Google!” (also fuck you for disabling audio/video control on chrome mobile from JS, EXCEPT for google.com/youtube.com etc)

Ummm... Html5 audio/video api works just fine on chrome mobile.

IOS safari is the bad actor, there ;)

Ummm..., no?

In Chrome mobile you can only do for example .play() on an audio or video element, if you call it from an eventListener that reacted to an onClick event.

Well, yes. You have to initiate things that could cause bandwidth use from a user interaction instead of, say, a timeout or an XHR or a DOMContentLoaded. youtube follows the same rules as everywhere else: it doesn't autoplay. Unless you're unwittingly opening youtube urls in the app instead of chrome.

iOS on the other hand actually -does- initiate bandwidth use on a <video> element, even if no interaction has taken place. If you watch Charles during an iOS video session, you will see 2-3 HTTP GETs before you even initiate playback. This can play merry hell if with your HTTP logs, if you pay attention to that sort of thing.

iOS also has major issues with multiple <video> elements; <video> must be full-screen on non-tablet i-devices (you cannot create custom controls); you cannot control volume via the user interface; it uses a nonstandard progression for its status-related events... (and sometimes they're just plain missing)...

I've spent the past few years working on web media players for mobile and desktop ;) I kind of know what I'm talking about. And due to its position in the marketplace, iOS Safari has stagnated - it's easier to get HTML5 multimedia working correctly in IE11 than iOS Safari. iOS Safari is the new IE6.

Perhaps this would be a good time to advocate link shorteners that use the standard HTTP redirect mechanism (bit.ly is one of them), so they can be HEAD'ed, and not some additional layer of obfuscation either via meta-redirect or (even worse) bury the target in some minified and obfuscated JS.

They don't want that because they don't want them to be automatically resolved so they can do metrics which is like 99% of the reason they are used in electronic form.

If they are temporary redirects, so not cached, they can do metrics from the server logs perfectly well.

Not if the intermediary, say, Twitter, de-shortens the link once, and then sends all the real traffic to the cached destination URLS. Not that they do that at the moment.

I have an IRCbot that is supposed to resolve URLs and find out if they ever end up on YouTube, and then post the video title, length and viewcount.

It was an extreme hassle to implement all of this, I am partially parsing JS and the DOM just to get all those redirect services to work. It’s a huge hacky mess, but the only fast working solution.

And, for the fifth time today, I can sincerely say “Fuck you, Google!”

Bitly has confirmed that this issue is resolved: https://twitter.com/Bitly/status/526012694835757057

Every bit of technology provides benefits and threats. URL shorteners are no exception: they definitely add value (and I don't mean just from the sender's perspective, but also on the recipient's end, eg. by allowing you to customize web addresses making it easier for people to remember them) but can also lead to harm (obfuscating links to malware).

It's like saying a hammer is a useless tool because it can bust your thumb.

if you're curious about following a bit.ly link, simply append a + to the end of the url and you'll be taken to that link's statistics page, which will also display the full link you're being forwarded to. a lot of other shorteners employ the same feature.

but obviously, just use an unshortener extension for peace of mind.

Can't this just be bit.ly links to unsafe sites? That dosen't make bit.ly unsafe as a whole.

agree, it's a bit like saying emails from unverified senders are a threat because they may be malicious. true but obvious

Yes it does, as you cannot sanely identify what the link is first.

Just FYI, you can append a + to a "bitlink" to preview it's url and see some stats. http://support.bitly.com/knowledgebase/articles/136551-can-i...

Sure. I said "sanely".

Yeah, like that's obvious or well-known.

Google is definitely not a fan. I send a lot of email each month (it's my business) and having bit.ly links in mail is a fast track to Gmail's spam folder or, in some cases, being rejected entirely. (Note: I'm talking bulk mail, not personal mail.)

I hate getting emails with proxied links, bit.ly included, because it's an in-your-face "I want to track you" statement. Never open them and unsubscribe after a couple of strikes. Your audience may be different, but I can't say I disagree with Google junk'ing your emails.

They're not junking my mails anymore because I'm not stupid enough to include bit.ly links ;-) However, clicks are still tracked in the majority of non-personal email nowadays, mostly using the click tracking provided by the large mail service providers, it's just not bit.ly.

Why would you use bit.ly links in email?

to track whether people open them or not

Because they want to track clicks on links to URLs that they don't control the content of. Otherwise they'd be using utm_ query parameters and an in-site analytics system.

Bitly links in email are super super shady.

Are they though? They could just set up their own redirection service on their own domain - not anything complicated in that. My guess is they use it because it's just a simple way for marketing to add some analytics - that doesn't have to be shady.

That's what parent said. I don't think he is implying it's shady behaviour just that it looks that way. Difference is it hides the real domain from the user and email scanners.

To signal spam content ;)

The sibling replies are mostly a load of bollocks. E-mail based publishers track clicks anyway through much better means, such as Mailchimp's click tracking - no reputable sender needs to use bit.ly to track stuff.

The reason bit.ly links can end up in mails is if third parties want to track clicks separate from the sender and the sender isn't smart enough to catch it. For example, people who want to include a job listing or some sort of sponsorship or advert in your mail. They'll often use bit.ly, simply because it works fine on Twitter, without realizing it's not a great idea in mail.

Above you said that you didn't sent spam because your recipients opted in. OK. Now you say that these email newsletters contain third-party advertising. You're sending paid advertising in an email. I would consider that to be spam, even if I opted in to your newsletter. I accept the fact that not everyone would define it that way, but I hope you see my point.

So advertising is now considered "spam"? Why do you think sites have newsletters?

If you go to a web site and there is paid advertising on the web site, do you consider the web site spam? You opted in to the web site by going to it and you can always not go back. You opted in to the newsletter by subscribing and can always unsubscribe if the total content isn't valuable enough. It's the same thing.

I see your point in so far as that it's fine to come up with and use one's own definitions for words, although every widely accepted definition of e-mail "spam" includes a criterion of being unsolicited. For example, in:

* law (e.g. CAN-SPAM or the Privacy and Electronic Communications Regulations 2003) - indeed CAN-SPAM even stands for Controlling the Assault of Non-Solicited Pornography And Marketing Act and reflects the US government's position, the EU's regulations refer to 'unsolicited communications' - http://www.legislation.gov.uk/uksi/2003/2426/regulation/22/m...

* encyclopedia (say, Wikipedia - http://en.wikipedia.org/wiki/Spamming - or the Encyclopedia Britannica - http://www.britannica.com/EBchecked/topic/941678/spam)

* dictionaries (say, Websters - http://www.merriam-webster.com/dictionary/spam - or the Oxford - http://www.oxforddictionaries.com/definition/english/spam)

* technical standards (say, RFC2505 - http://tools.ietf.org/html/rfc2505)

* the viewpoint of people extremely against spam, such as Spamhaus - http://www.spamhaus.org/consumer/definition/ or SpamCop - http://www.spamcop.net/fom-serve/cache/14.html

* the terms and conditions of e-mail service providers and senders - e.g. http://mailchimp.com/legal/terms/

Some of the replies you got with knee-jerk reactions about spam are truly ridiculous. People are just spouting nonsense without any idea about what you do. I hope you aren't letting them get to you. I enjoy your newsletters.

I would say Google is doing the smart thing in your case. I refuse to click on any link in an emailer that is obfuscated.

If you're sending spam, it's only appropriate that it goes in the spam folder.

It seems to me that google is doing a good job, here.

If someone's sending spam, sure. I didn't say a single thing about sending spam, however. I send mail to 200,000 subscribers who double opted in and want to receive the mail :-)

> I send mail to 200,000 subscribers who double opted in and want to receive the mail

Google has no way of knowing that.

They do. It is called IP reputation, and is the main reason your inbox is mostly spam-free: Google (and all the other large mail providers) keep reputation scores for IPs / IP blocks, based on what volume of ham/spam they send, and for the larger commercial senders they outsource anti-spam management by saying "Hey MSA: here are the minimum requirements for email getting relayed to Google customers. Ensure your customers respect them, or we'll end you. Toodles!"

> They do. It is called IP reputation

I didn't say Google has no way of detecting spam - I said Google doesn't know the recipients "double opted in and want to receive the mail"

Again, this is ensured by outsourcing verification. It is the same principal as "How does the USFG know that an arbitrary bank account in Japan is not controlled by terrorists and hence doesn't require enhanced scrutiny for wiring money to the U.S.?" The USFG trusts that the Japanese financial regulator has equally stringent regs, which will severely consequence a bank if they let a terrorist bank there.

How does Google know you're clean on permission if you are coming from a MailChimp IP? Because Google knows MailChimp knows what Google will do to MailChimp if you aren't innocent.

definitely not spam. i subscribe to 3 of peter cooper's weekly newsletters and they're immensely useful to me as a developer.

Eh, frankly it just seems like its google slipping up. If only a few hundred bitly links have viruses I'm impressed (I'm sure there are more but the percentage still doesn't seem that bad).

All Bitly links are no longer blocked by Google Safebrowsing in Chrome or Firefox.

Not using google but also safari warns about malware via google some how.

That's because Safari is using "Google Safe Browsing Service" - You can toggle that in Preferences/Security.

bit.ly is working fine for me currently but fb.me is currently being blocked

For now most shortener services give warnings. Internet is broken for next half day.

I feel like Google does this many times to hurt the competition.

What is "this" in this context?

Scan websites for vulnerabilities?

Make the results of automated vulnerability scans available?

makes it difficult for website operators to exist like bit.ly to exist by warning users that there are potential risks, when the risks are almost the same if not more with their own, similar services.

Microsoft could swoop in and solve this :D https://www.noip.com/blog/2014/06/30/ips-formal-statement-mi...

Reply on news.ycombinator.com