Hacker News Reader

> Would Google discourage this?

Yes, I think so. When ISPs wildcard all domains, I think the typical reason is so that when you've typed an invalid name, they return a page with search results guessing at your intent. If I'm right (I might not be), then Google has plenty of incentive to stop that from happening, or at least know that it is happening.

First, those pages with search results are often ugly, full of ads, poorly formatted, etc. Google would like you to have a nice experience in Chrome so that you're a happy user. If you see an ugly, messy ad-ridden page (especially if the ads aren't Google ads) every time you make a typing mistake, Google loses out.

There are probably other more subtle and nuanced reasons that Google would dislike that practice, but I'm not really sure of specifics. (In fact, I'm not entirely sure that DNS wildcarding is even related to those ISP search pages...but I think it is.)

Also, if the ISP is answering every DNS query with "yes this is a valid domain" it breaks Chrome's domain discovery feature, described a bit further down the SANS article.

I don't think that DNS wildcarding has anything to do with it. They just set up their DNS server to return their own address instead of NXDOMAIN when the domain doesn't exist:

http://en.wikipedia.org/wiki/DNS_hijacking

  I have a question. Why would wilcard domains be 'nasty'?
  It's not that unusual, is it? Would Google discourage this?

You're thinking of wildcard subdomains, where a [[domain.com]]'s naming servers are configured so *.domain.com resolves.

Google is taking action against wildcard domains, where the nameserver resolves every domain to some IP. Valid domains are (usually) resolved to the correct IP, invalid domains are resolved to the ISP's ad-laden search page.

exactly. it's as if your phone company were to reroute calls to non-existent numbers to toll number instead of the usual intercept message "the number you dialed is not in service." the catchall search/ads page wastes way more bandwidth and DNS lookups than NXDOMAIN.

Hmmm.... maybe I should change my setup then, and somehow fill the DNS server with names from the database, instead of relying on a wildcard. Thanks for the insight!

I think you're still misinterpreting it a little bit. Google won't penalize you for using wildcards on your domain. They only look for ISPs that use wildcards as a "catch all" to show their branded search page.

Have a look at this scenario. This scenario has nothing to do with Google or Chrome (yet).

Assume: User is using their ISP's DNS service, and the ISP uses DNS wildcards to redirect users to a branded search page.

1) User types notifyo.com in to a web browser; a misspelling of notifo.com

2) ISP DNS finds no match for notifyo.com upstream, so returns their own web server's ip

3) User is shown a web page with search results when they really wanted a specific website

ISP's do this by specifying wild cards at the TLD level. This means .com, .net, .org, etc will all return the IP of their search servers if nothing else matches.

Google Chrome checks for these wildcards only. It does not check for wildcards on your domain. For example, Google Chrome would not find, nor object to .notifyo.com.

Personally, I find the use of wildcards at the TLD level to be flat out offensive. It's a break in the basic protocols of the internet. If a website doesn't exist, I need to know. I don't want a search page, I want an error that reflects the status of the domain I have attempted to visit. That's why I use the 4.2.2.x DNS servers.

Have you seen the 8.8.8.8 and 8.8.4.4 DNS servers? (http://code.google.com/speed/public-dns/docs/using.html)

I have. Unfortunately, they're 2-3 times slower than the 4.2.2.x servers for me, depending upon cache status:

    # first request to google
    $ dig @8.8.8.8 www.bradlanders.com

    ; <<>> DiG 9.6.0-APPLE-P2 <<>> @8.8.8.8 www.bradlanders.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20999
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;www.bradlanders.com.		IN	A

    ;; ANSWER SECTION:
    www.bradlanders.com.	14400	IN	A	69.163.164.23

    ;; Query time: 272 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Thu Jan 27 16:01:23 2011
    ;; MSG SIZE  rcvd: 53

    # first request to L3
    $ dig @4.2.2.2 www.bradlanders.com

    ; <<>> DiG 9.6.0-APPLE-P2 <<>> @4.2.2.2 www.bradlanders.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62436
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;www.bradlanders.com.		IN	A

    ;; ANSWER SECTION:
    www.bradlanders.com.	14400	IN	A	69.163.164.23

    ;; Query time: 111 msec
    ;; SERVER: 4.2.2.2#53(4.2.2.2)
    ;; WHEN: Thu Jan 27 16:01:31 2011
    ;; MSG SIZE  rcvd: 53

    # second request to google
    $ dig @8.8.8.8 www.bradlanders.com

    ; <<>> DiG 9.6.0-APPLE-P2 <<>> @8.8.8.8 www.bradlanders.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58197
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;www.bradlanders.com.		IN	A

    ;; ANSWER SECTION:
    www.bradlanders.com.	14400	IN	A	69.163.164.23

    ;; Query time: 171 msec
    ;; SERVER: 8.8.8.8#53(8.8.8.8)
    ;; WHEN: Thu Jan 27 16:02:47 2011
    ;; MSG SIZE  rcvd: 53

    # second request to L3
    $ dig @4.2.2.2 www.bradlanders.com

    ; <<>> DiG 9.6.0-APPLE-P2 <<>> @4.2.2.2 www.bradlanders.com
    ; (1 server found)
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7408
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; QUESTION SECTION:
    ;www.bradlanders.com.		IN	A

    ;; ANSWER SECTION:
    www.bradlanders.com.	14317	IN	A	69.163.164.23

    ;; Query time: 35 msec
    ;; SERVER: 4.2.2.2#53(4.2.2.2)
    ;; WHEN: Thu Jan 27 16:02:54 2011
    ;; MSG SIZE  rcvd: 53

It's nasty when the ISP returns an A record for an advertising filled spam page instead of NXDOMAIN.

Wildcard domains (*.bloggo.com) aren't nasty. Wildcard DNS servers are. They inject domain name data into zones owned by other entities; if you type news.ycombinator.com, you get this server, but nwes.ycombinator.com gives me a JS redirect to www.ayudaenlabusqueda.com.ar/?query=nwes.

This is what I gathered from the article, that it is looking for evidence of malware injecting itself between your browser and the DNS lookup, or poorly configured DNS servers doing similar.

On a similar thread, Googlebot will often request a page that it can safely assume will 404, like example.com/noexist_d6956f303b25361a.html, just to make sure that non existant pages actually return a 404 instead of a 301 to a page with dynamically generated content based on the request name.

This is interesting news. My use case is indeed having many subdomains together with a wildcard DNS like '*.somesite.com', and I'm not designing some nefarious plot to redirect 'notexistant.somesite.com' to an ad-serving site. Instead, a page saying 'this subdomain can't be found', together with a registration form, would be shown.

(Speculation mode on) Would Google penalize this setup with a lower Pagerank?