Saved 10 billion DNS queries per month by disabling DNS Prefetching

Whoever it was that fixed the XSS (necro?), I'm impressed how fast that was.

But please don't rely on just escaping < and >. You have to worry about double-quotes too, I can end a string and add a "onload" or "onfocus" attribute if it's already in a tag. And sometimes you have to worry about single quotes. In fact, there's a lot to take a look at.

Instead of just fixing the case at hand, try to be proactive about it. Check to make sure you don't have anything else.

Edit: Click the search box, for example. http://www.pinkbike.com/forum/search/?q=%22%20onclick=%22ale...

One reason to do per-user domains is to prevent cookie leakage caused by user-specified content. A few years ago LiveJournal got bitten by a firefox bug that allowed JS execution inside CSS. Since they allowed each user to style their own page that meant that a malicious user could make the CSS at "www.livejournal.com/~evil_user" capture the viewer's www.livejournal.com cookies. That's why they moved everybody to username.livejournal.com, which previously had been a pay-only feature.

I think the really perplexing about this article is why browsers are so aggressive with DNS prefetching. If there are links on the page to 300 different domains, is there much benefit to precaching ALL of them? It seems clear that very few of the domains are likely to be clicked on. If anything they're probably hurting performance by bombarding the local nameserver with a flood of requests.

It sounds like browsers need a better precaching heuristic.

It's a good point, and in retrospect I may not have used the username.example.com pattern, but this was a choice made may years ago. Users seem to like this, and unfortunately these recent changes in browsers make this pattern less desirable. Some other sites that do this subdomain style are the likes of tumblr.

The thing to note is that implementation of browser prefetching is further putting DNS requests out of the control of the publishers. 10 billion queries was something like 5000 dns req/s and that is considerable resources of some dns service. I wish dns was not per query costs, but it would seem that is the metric that resources would be based on. The point of the article is to let people know that if you fall into this dns pattern, it maybe a result of the recent prefetching done by browsers, and you may have a way to resolve the issue.

Perhaps a per-query charge isn't optimal, but how else can a provider of a service equate supply with demand?

More load implies additional cost, and DNS providers are no exception; there is an incremental cost of serving a DNS response. Granted, it's small, but real nonetheless.

One could charge based on bytes transferred instead, but charging for bytes is functionally equivalent to charging for responses.

I prefer HTTP headers instead of meta tags where possible.

An example solution at the load balancer level, assuming the use of Varnish:

In vcl_fetch:

  if (req.url ~ "\.(htm|html|php)" || req.url ~ "\/(\?.*)?$") {
    set beresp.http.disable-dns-prefetch = "1";
  }

In vcl_deliver:

  if (resp.http.disable-dns-prefetch) {
    remove resp.http.disable-dns-prefetch;
    set resp.http.X-DNS-Prefetch-Control = "off";
  }

Not closing the tag is nothing specific to HTML5, it just means you aren't strict on XHTML. HTML4 and others would be the same.

Does Google take DNS prefetching into account when they measure the speed of a web site?

See my reply here: http://news.ycombinator.com/item?id=2306788

He is creating thousands of unique subdomains, so nobody can cache anything.

When I worked on Akamai's DNS server, I was told that most ISPs cached much more aggressively than we told them to. We would set TTLs at...I want to say 5 seconds...because we were directing traffic based on real-time network conditions; but most ISPs would cache for at least an hour.

Exact numbers may be off, of course; it was a few years ago.

Depends on the location of your users. Quicker to have someone host it is multiple places all over the world (assuming you care about perceived page load speed)

But that is a good thing for me!

For those downvoting, did you want the server-side solution?

    <meta http-equiv="x-dns-prefetch-control" content="off" />

DNS wildcards are server-side, so there's no way for the client to know that the response they got is for *.domain.com but not www.domain.com.

No, it's not; the issue is that the domains prefetch REGARDLESS of whether the links are ever clicked on. The browser is pre-fetching in case the user DOES click on the link. This really verges on being a browser bug, esp. since in firefox's case, it fetches both A and AAAA records.

you have neglected to take into consideration the cost of running a hosted DNS. Costs for DNS service for any large traffic site is an important consideration.

In our (deviantART's) case, the 24-60 links to subdomains were set up so if you clicked them, they dynamically load with javascript on the client side, never actually hitting that subdomain. No time wasted on the user's side, since the browser was prefetching domains that were rarely being hit.