Hacker News Reader

Comment by _delirium | Hacker News Reader

To save an extra click, basically the only information in the stack overflow answer is: see http://code.google.com/web/ajaxcrawling/

View on HN

Right.

By the way, the whole idea of Ajax Crawling as defined per Google struck me as wholly unnecessary. If you are using unobtrusive JavaScript, and HTML like so:

    <a href="http://twitter.com/ev">@ev</a>

with JavaScript adding a click handler such that a user’s click on the above link actually only triggers an AJAX deal (FWIW, jQuery stub:)

    jQuery('a[href^=http://twitter.com/]').live('click', function(){
        // Do AJAXy thing instead, then
        return false;
     });

Then, assuming you serve up essentially the same content at /ev that you would display via AJAX, you have simultaneously just enabled your site for not only modern browsers, but also robots and dumb phone / JS-avoiding users. (And middle-clickers.) No goofy API required.

Edit I just realized, if you have links to `http://twitter.com/#!ev` in the wild, then the AJAX crawler thing becomes actually pretty useful.

Yeah, the links in the wild are the problem. Users have a habit of copying URLs from the URL bar, unfortunately.

I think Google could have done this better; they could have made it so all ajax URLs are crawlable (standard robots rules apply) as long as your site opts in, then you'd only have to support pretty URLs, something you probably have to do anyway.

For example, now twitter needs to support twitter.com/#!/ev and twitter.com/ev and twitter.com/?_escaped_fragment_=ev

Users have a habit of copying URLs from the URL bar, unfortunately.

How else are people supposed to bookmark or share an URL in your opinion?

That was a joke. Our site and many others have explicit ways to get a non-ajax URL for sharing, but it is of course more convenient and more natural in a browser to just grab from the URL bar. It's unfortunate for us but not remotely unexpected.

This is where I'd expect the <link rel="canonical" href="blah" /> element to be used. With browsers using this as the preferred url for a bookmark - and maybe a service for sharing or copying urls from the browser that was easy to use.

That would break bookmarking a section of a long article.

And how, pray tell, would you expect a crawler to fetch the content at twitter.com/#!/ev, considering that the server doesn't even see the hash part of the URL? That's the whole point.

The simple fact is that plenty of sites (like Twitter, Facebook, and lots of Google properties) are using client-side code to build fast interactive sites, and it necessitates this kind of infrastructure.

The hash is also the only way (AFAIK) to insert an Ajax request into the browser history, thus enabling the use of the browser's back button.

History API[1]? :) If History API is being used alongside with alanh's method, that would eliminate the links in the wild problem. Flickr's lightbox view[2] is using this method for its AJAX viewing.

The biggest holdback is Internet Explorer, though. (Well, just like almost everything else.)

[1]: http://www.whatwg.org/specs/web-apps/current-work/multipage/...

[2]: http://www.flickr.com/photos/cubagallery/3986524856/lightbox...

It also prevents the user's id from being leaked in the http referrer header

Reply on news.ycombinator.com