Hotlinking to jquery.com will be disabled on January 31, 2011 | Hacker News Reader

Hotlinking to jquery.com will be disabled on January 31, 2011

blog.jquery.com

150 pointsUncle_Sam97 comments

We did a lot of analysis of the sites that were actually hotlinking, and are planning to reach out to those that are above board and should know better. By and large, however, most of the sites that were hotlinking were porn/phishing/generally nefarious, which weighed heavily into our decision to pull the plug relatively soon. Serving people who need jQuery and jQuery UI documentation, etc., is a higher priority than not-pulling-the-rug-out-from-under networks of porn sites - people who have more than adequate access to their own hosting and distribution resources.

OTOH, TaylorSwift.com was also hotlinking, which is pretty awesome.

I am currently trying to talk to Taylor about sharing cdn resources.

If anyone is not aware, one of the best CDNs out there for getting jQuery is the Taylor Swift "TaylorNation" cdn.

http://cdn.thetaylornation.com/taylornation/resources/displa...

I encourage everyone to use it.

slexaxton imma let you finish... but amazon has one of the best cdns of all time

I like the one comment about serving up evil.js to hotlinkers instead (https://github.com/kitgoncharov/evil.js/blob/gh-pages/evil.j...).

How have I never heard about this hilarious script before?

Wow. That script is all kinds of awesome. Javascript is unconventional and unpredictable as it is, without intentionally hosing the entire runtime environment like this.

One of the reasons JavaScript crypto is a bad idea in most cases.

I lol'd @

    'PI': 3.2

For those of us who do not know javascript (a rarity on HN, I know) could you explain what it does?

It does... many things. All sort calls are hardcoded to return [4, 8, 15, 16, 23, 42], regardless of input. It scrambles the log bases for the math logarithm commands, it hardcodes the output of the upper-casing string call to a constant, and what I've reported is just a sampler. It is very well-named.

Could be improved a bit, though. Many of its entries lack subtlety. If you really want to be evil, make uppercase do something like uppercase all but one letter, or "uppercase" the digit 1 into exclamation marks, 2 into at signs, etc., convincing the poor developer that the JS upper case function is broken and affect digits. ("Must be unicode or something.")

How would this not start a backlash of "JQuery (or whatever) is broken."? If the only indication is that a hotlinked .js suddenly stops working, what exactly is the debugging process there? I simply don't think most web developers are going to find a useful path from "scrambled log bases" to "hotlinked js," if they even identify "scrambled log bases."

Then again, maybe the point is to muddy your own waters? I'm reminded of an aphorism dealing with activities that should not be undertaken near one's place of slumber.

I don't think it's a serious suggestion that evil.js be included, just a funny thought.

Ah, dopey me!

From my inexperience (3 years at one company, 1 year at another), the sad truth appears to be that the developers would care just enough to hide the problem.

For example at my last job, I was pair programming with the "most valuable" C++ programmer. We were debugging one of his monolithic heightmap classes. Some top-level method was calling some deeper method that was doing a lot of different complicated things. After examining the top level method in the debugger, and looking at the results after calling the deeper method, he wrote some code to "patch over" the problem. I.e. The deeper method was causing the results to be bad in some way, and his solution was to add code to overwrite the bad results with the expected ones. Then we ran the program and it worked.

To my surprise, he stopped there and committed the code. As he was writing the commit message, I asked "Hmm, aren't you interested in why the internals don't seem to be working properly?" He shrugged and said "No."

I hear their game engine has suffered rather severe performance problems and general bugginess. It also looks visually to be 2004-era.

  Many of its entries lack subtlety.

An Underhanded Javascript contest would be amusing.

It's actually fairly readable if you have a background in coding at all. Here are some clues:

1. Every built in in Javascript can be overwritten.

2. If you see something like ObjectName.prototype.fnName, then what you're doing is defining a new fnName function on ObjectName. The thing about Javascript is that you can add new functions to objects, and since everything can be overwritten, you can also replace old functions with new ones. So when you do something like Array.prototype.sort = function(){...} then you're overwriting the builtin sort method on Arrays. Most of the script is just defining builtins to have bogus results.

3. Javascript has a built in Math object, like a few other languages. The language also allows you to create objects on the fly with bracket notation: the layout is key: value. So the Math definition defines Math.PI to be 3.2.

4. There are a few other jokes in there too (like defining alert, which makes a message box, to be eval, which evaluates the string).

Crockford used to prevent hotlinking to his JSON library (its now on github) in an interesting way: right at the top there was an alert() line that you had to remove before using.

This would probably be the best way to transition. If they add an alert to their library and leave it up a week, most people should notice and fix it.

If Crockford used it in production though, the alert() would be removed there, right? So someone could still in theory hotlink that.

Unless jquery.com itself uses Google's CDN (which it already is for some of its javascript)

    if (location.host.search(/crockford.com$/i) != -1) {
        alert("Stop hotlinking me!")
    }

I don't think you can fool that, but I'd love to hear about how I'm wrong.

window.alert = function () {};

    if (location.host.search(/crockford.com$/i) != -1) {
        throw "Stop hotlinking me!";
    }

You code should read != 0 to have the desired effect.

And even then, this can bypass it:

  var temp = String.prototype.search;
  String.prototype.search = function() { return 0};

    if (location.host.search(/crockford.com$/i) != 0) {
        throw "Stop hotlinking me!";
    }

  String.prototype.search = temp;

Though that may cause other problems.

My point was that it's just a game of cat and mouse. I could come up with lots of workarounds for almost anything you throw at me. Example:

    var expectedHost = "crockford.com";
    if (expectedHost.length !== location.host)
        throw "Stop hotlinking me!";
    for (var i = 0; i < expectedHost.length; i++)
        if (location.host[i] === expectedHost[i])
            throw "Stop hotlinking me!";

(though it string[x] might not work in every browser)

So thanks for further demonstrating my point!

But really, just check the referrer header.

Point. I was unclear in my phrasing, I was more looking into just not accidentally bypassing it. Deliberately bypassing it probably can't be stopped but really at that point you've already lost. Given what I've seen in the world somebody grabbing an existing proxy script and regexing out the check and never once stopping to think this is way worse than hosting the file yourself wouldn't even make me blink.

You can definitely fool that: http://www.adambarth.com/papers/2009/adida-barth-jackson.pdf

  <script>
  var oldAlertFunction = window.alert;
  window.alert = function(){};
  </script>

  <script src="crockford.com/json.library.js"></script>

Arms race, but just sayin'

The point would be to give webmasters a heads-up and time to fix their sites before hotlinking is disabled, not so much to prevent them from using the file.

Indeed. I was just being silly :)

And if you want to avoid changing the rest of the page:

  <script src="crockford.com/json.library.js"></script>

Anyone smart enough to think of that would be smart enough not to hotlink

Frankly I'm surprised jquery even allowed this to begin with. Now there will be sites that rely on this tha break, some of which won't be fixed for ages (if ever).

I guess I shouldn't be but I'm still surprised people would even do this given that Google s offering the service for free. Hotlinking has always been antisocial.

Never doubt the laziness of people, especially other developers.

I'm actually more surprised that the site didn't either use Google's API to host the download, or provide a hotlink button right there.

Some might be from laziness, but I would imagine most of the hotlinking is from ignorance.

I guess it is both: Too lazy to find out the right way to do it.

Agreed. I'll bet a significant portion of it is coming from cut-n-pasted html where the users don't even realise they're using hotlinked javascript, and have probably never even heard of jquery (and, unfortunately, are spectacularly unlikely to hear about this change on jquery's hotlinking policy).

Always seemed to be to be a bad practice from a security point of view - you are putting the security of your site in the hands of whoever is hosting the .js

This also applies when .js is dynamically included as a type of API call to embed widgets and whatnot - but in those cases there's a necessary reason - it's the only practical way - but for a simple .js, you should be managing your own .js library and publishing on your own (including all the speedup tricks you know you should be doing)

> Always seemed to be to be a bad practice from a security point of view

This is why I can't understand people's recommendation to use Google's or Microsoft's CDN. Even if you assume they're not going to be malicious, you have to trust that they're secure. Not to mention that the CDN owner can derive accurate traffic stats from the number of requests for the JavaScript.

Why would I be less sure that they're secure than my own personal hosting?

No reason you would be. But you can be sure that your personal hosting is more secure than (your personal hosting ∪ Google). In the latter case if either breaks you're screwed.

You know the components of, and the capacity to upgrade, your own stack. With hotlinking, you (and consequently your users/customers) have an external dependency to a mysterious system with who-knows-what going on.

As long as the CDN is configured to transmit its files with a far-future expires header (which Google and Microsoft's CDNs are), the CDN owner can't accurately track usage at all.

In a perfect world, the CDN would only handle one HTTP request per user per expires period (+1 year on Google and Microsoft's jQuery CDN). In reality, the file will be pushed out of or cleared from caches for various reasons, but they still persist long enough to make tracking end-usage of the CDN meaningless.

Performance. Many of your visitors will already have it in their cache.

My concern has always been the one-day-in-a-thousand when the CDN will be down and so my sites won't work. When my sites go down it doesn't matter as the entire package stops working, but I'd hate myself ten times more if things stopped working due to an admittedly extremely rare CDN failure.

That said, I'm open to the idea that I'm being extremely foolish about this ;)

Most users will have the javascript in their cache anyway, so they won't even be hitting the CDN. Also, why not check to see if the object exists before doing anything else?

    <script id="jquery" src="http://code.google.com/jquery.js"></script>
    <script>
    if (jQuery == undefined) {
        document.querySelector("#jquery").src = 'Backup CDN';
    }
    </script>

That should work, in theory, if not then just remove script#jquery, create a new one and append to the document before executing any script(s).

It's at times like this that I feel really stupid ;) Thanks, you can guess what I'll be changing on my sites the minute I sobered up, err, sometime in the next couple of days ;)

iirc changing the src attribute directly does not resend the request (in at least some browsers). you need to create a new node.

Stolen from the html5 boilerplate

    <!-- Grab Google CDN's jQuery. fall back to local if necessary -->
    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.js"></script>
    <script>!window.jQuery && document.write(unescape('%3Cscript src="js/libs/jquery-1.4.2.js"%3E%3C/script%3E'))</script>

Superb, thank you! I'm more feeling significantly more enlightened :-D

Plenty of people use Google Analytics, which, of course, gives Google just as much access.

I trust Google and Microsoft enough for the benefits I get in return (faster loading clients). Sure they can screw me over but not too badly. Worth the risk for some applications.

Perhaps a little harsh. Is the problem bandwidth or connection quantity? If it's bandwidth then they should just 301 to the Google CDN.

They are not disabling their CDN. They are only disabling hotlinking against other sources. This was in the first sentence.

I understand that. The question still stands - if it's a bandwidth problem on jquery.com, why not just 301 it somewhere else?

Because 301'ing provides little or no incentive for the offenders to do the right thing. Also it doesn't stop the excess traffic, it just reduces it.

Straight connection overloading. We're already redirecting the majority of the files - but somehow the hotlking continue to persist.

Makes sense, good decision then.

How are you preventing hotlinking while still allowing the files to function normally on jquery.com? Inspecting the referrer or checking to see if other resources were previously downloaded?

You can do it in your .htaccess file. Define some extensions that are blacklisted and redirect referrers outside your own domain to a place of your choice.

CPanel allows you to do this directly (although you might want to back up your .htaccess file first).

Now is probably a good time for a reminder: http://scriptsrc.net/

(Up to date CDN links for a bunch of javascript libraries, including jquery)

Damn shame. I used to hotlink during testing when I was too lazy to download the actual JS and host it. It was a nice way to build wicked fast little sites.

Just hotlink to Google's version.

Ways to solve the problem once and for all:

- Listing the sites that hotlink it would be a nice idea.

- Changing the script on the hotlinked files to pop-up a warning that the site is doing something improper and urging people to contact the owner.

- When that doesn't work, break the sites.

When the change is made, changing the results of the HTTP GET to some javascipt comments explaining what happened and pointing them to some CDNs which offer the same service would be a good idea.

The jquery people are being incredibly generous. Others would have just tweaked the JS to produce some shock site or text to the effect "you are a bandwidth thief".

It should be trivial for them to scan their access logs for referrers, and then send a mailshot out to each domains webmaster address.

To me, that seems like the polite thing to do.

They are already going way above and beyond by extending their deadline to the end of January. They have the capacity to be much ruder about it.

Hotlinking like this has been a well known web evil for well over a decade. That people still do it and just hope for the best is indicative of extraordinary laziness and selfishness.

You're attributing malice to something which could be attributed to ignorance. A common error.

They could just break all those sites without giving them personalised warnings, yes. And they would also be completely entitled to yes. But for the small amount of effort needed, why not try to minimise the damage.

Malice? As quaint as that saying is, malice plays no part in this discussion.

I'm attributing laziness, which is quite removed from malice. People using jquery should be evolved enough to know what CDNs are, and the dangers of hotlinking (which is as simple as "jQuery decided to reorganize their site, and now a thousand sites are broken").

I guarantee that some people are using the wrong URLs because they received bad advice, or copied from code they shouldn't have, or were simply told by somebody that they could link to the version on jquery.com, so did.

These people don't need punishing, they need educating. I find the whole "fuck em, it's their fault" attitude to be very childish.

"I guarantee that some people are using the wrong URLs because they received bad advice, or copied from code they shouldn't have, or were simply told by somebody that they could link to the version on jquery.com, so did."

Is that not precisely what I said? Laziness.

I feel like I'm debating with a tired Hallmark card. Yes, high road and all that. We get it.

Just as you misplaced the malice quote (you were a little too eager to play that one), now you're on about punishment: Who said anything about punishment? Saying "Don't hotlink, but we're giving you 30 days grace" is not remotely "punishment". That you portray it as such is, honestly, outrageous.

"Is that not precisely what I said? Laziness."

No. You can't both understand the word "laziness" and attribute that to the explanations I provided.

"I feel like I'm debating with a tired Hallmark card. Yes, high road and all that. We get it."

I feel like I'm debating with somebody who is very intolerant.

"Just as you misplaced the malice quote (you were a little too eager to play that one)"

I did not misplace it, and I stand by the statement, and exactly how I used and formatted it.

"now you're on about punishment: Who said anything about punishment? Saying "Don't hotlink, but we're giving you 30 days grace" is not remotely "punishment". That you portray it as such is, honestly, outrageous."

You're easily angered aren't you. As I said, it would be trivial to contact them directly and tell them about the problem, and not doing so when it would be so easy giving the reason that they should have known better in the first place is a clear kind of punishment yes.

I'm not continuing this discussion with you. I have more fun things to do with my New Years Eve than debate with an angry troll.

Pretend to take the high road. Misrepresent what I've said in reply after reply (now you're onto the "don't get angry!" screed which is one of my favorites). Pull the "I'm done!" card just as you've declared me a troll. I see that you've mastered this technique, and your black belt awaits.

I'm not done because I've debated with this ridiculous tactic enough to see it for what it is.

Copy-pasting code that you don't understand is pretty much the definition of laziness.

And just to be clear, it's "trivial" for jQuery to do this, in your mind, because you're not the one having to do it. Probably thousands of tiny sites that are doing this (I doubt it's any big numbers sites), each requiring you to go through the whois process to find a webmaster. Not as trivial as you proclaim.

> Copy-pasting code that you don't understand is pretty much the definition of laziness.

Disagree. Copy-pasting code that you don't understand is how most people learn how to code on the web. It's how I learned.

Most junior web developers do not understand everything that goes on in the web stack. They piece together the ability to run a functioning website by putting together bits of knowledge they pick up along the way. Some of this knowledge comes from reputable sources, like the jQuery documentation. Some of it comes from sources that may, albeit wrongly, recommend hotlinking, such as friends or Experts-Exchange.

When you're new to a subject, and there are that many unknowns, and you don't have a great mentor, it's hard to know exactly which parts are important to understand thoroughly, and which can be attributed to "magic" for the moment. Most people learn to program for the web because they need to get a project online, which means they are more focused on getting something working, rather than trying to grok every bit of the stack. I would bet that the vast majority of people that hotlink jQuery fall firmly into this category. It's not that they don't want to learn how not to hotlink, or are too busy or too lazy to learn, it's that they don't know that this is something they need to learn.

And that, sir, is definitively ignorance, not laziness. To be precise, it's ignorance of the fact that they are ignorant.

PS: Don't take disagreement personally, and don't call the person you're disagreeing with a "Hallmark card". It makes you look like a troll.

Dude, don't feed the trolls.

This is the problem. I did not "take the high road", I did not "misrepresent" anything you've said. The angry comment was made because you said that what I said was "outrageous". I mean, seriously? I did consider that you were trolling, and I did leave because I had better things to do.

How does one exit an argument with somebody who is behaving so irrationally, without getting accused of all that bullshit?

Here's how I read it:

I guarantee that some people are using the wrong URLs because they received bad advice

Didn't doublecheck the advice: lazy.

or copied from code they shouldn't have

Lazy coding.

were simply told by somebody that they could link to the version on jquery.com, so did.

See #1.

Not double checking all advice, is lazy? There isn't a human alive who has double checked every piece of advice they've received.

Copying code is "lazy coding"? There isn't a programmer alive who hasn't done this.

So these people are guilty of being alive and being programmers? Yeah, lets break their websites! That'll teach them!

This is the wrong attitude. The professional attitude is to at least try to resolve your problems without fucking other people over.

> People using jquery should be evolved enough to know what > CDNs are

There are lots and lots of monkeys using jquery. Cargo-cultist monkeys, at that. At least that's my impression from the browser bugs I've seen filed with snippets of jquery code.

I wouldn't extrapolate knowledge of CDNs, or much of anything other than jquery basics, from use of jquery.

> Hotlinking like this has been a well known web evil for well over a decade.

Except where the site expressly permits it.

It should be trivial for them to ... send a mailshot out to each ... webmaster address

Well indeed, but even if you ignore the time taken to do WHOIS lookups and presume that they'll go to the right person (e.g. blogs hosted on Wordpress.com) you're still ignoring the serious amount of computing power that it takes to send a significant number of emails.

I was suggesting webmaster@domain for each of the domains. Not 100% full proof, but no method is.

I'm assuming we're talking about thousands of emails here, not millions.

I'm assuming we're talking about thousands of emails here, not millions.

When was the last time you tried sending thousands of emails? It's a surprisingly non-trivial task to send more than a few hundred emails - and that's ignoring setting off spam filters and getting IP addresses blacklisted.

Speaking honestly, I like your idea, but it's my opinion that if it were that simple then the jQuery guys would do it. Since it isn't that simple, they're not.

You're saying this to the wrong person. I run mail systems for a living. IMO, it is a trivial task to send tens of thousands of emails.

It should take about a minute to craft a one liner to pull the relevant domains out of the access logs. It should then take maybe half an hour or less to write a batch script to queue up the emails.

Then sit back and watch them go... What else is there?

You're saying this to the wrong person. I run mail systems for a living. IMO, it is a trivial task to send tens of thousands of emails.

What hardware are you running that supports sending out tens of thousands of emails, handling the inevitable DNS errors, timeouts, bounces (yes yes, I've read about the RFC ;)) and that manages all of this in a reasonable amount of time? Genuine question, I'm curious.

If we're only discussing tens of thousands of emails, I'd probably get the cheapest Linode available and bung Exim on it. The average home PC could handle this without issue...

Shouldn't need to do whois lookups, webmaster@ is required by RFC.

http://www.faqs.org/rfcs/rfc2068.html

Well, then. They can just report any domains that bounce webmaster@domain to the local authorities, and a round of arrests and/or fines will definitely be in order.

rfc-ignorant.org is all over that already.

In theory yes. However since that address gets a lot of spam, it's quite unlikely that that email address is looked at. RFC are a good guide, but the community has mostly dropped that webmaster@ convention.

Really? I thought it was still common in the community. Why do you think otherwise?

The only occurrence I can find of the word webmaster on that page is using it in an example of a client setting the From request header.

Just for the record, here are the CDN versions:

http://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js (always the latest version)

http://code.jquery.com/jquery-1.4.4.min.js

http://ajax.microsoft.com/ajax/jQuery/jquery-1.4.4.min.js

Avoid using the "latest version" reference. In order to ensure that it's serving the latest version, it serves the script with an extremely short expires header.

Not to mention that referencing the latest version means that your code will be untested against anything but the library version you built against.

You forgot: http://cdn.thetaylornation.com/taylornation/resources/displa...

Keep in mine that many sites use the Google version, so if you do to, some of your visitors will get it "free" (from their local cache). The caveat about not using the "latest version" stands. And for the gods' sake don't use the actual google Ajax loader API, it's an extra script to load. (Include the above URL directly, not by doing a .load() on something else.)

Reply on news.ycombinator.com