Hacker News Reader

Comment by maccman | Hacker News Reader

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h...

Your second point has merit. However I'd counter it thus: * You can load a HTML page first, containing a representation of the end content, so you can avoid the flash * You can show some sort of loading indicator on page load. Users are usually fine with waiting for the initial page load - it's only subsequent interactions that need be fast.

In other words, there's no reason why the page need be blank.

View on HN

Only Google does that currently.

And while people can build crawlers that can fetch Javascript rendered pages, these scripts will have to drive real browsers with rendering engines, not just simple HTTP clients. And this is a problem, because fetching millions of web pages will require serious infrastructure work, versus a crawler using libev that can fetch dozens of pages in parallel without a sweat.

Also, that spec you gave is not exactly how Google operates. That's the advice Google gives for making web apps compliant, but the reality is that Google is smarter than that and it takes Google a good amount of effort to fix other people's screwups.

Accessible to crawlers doesn't mean Google-only and you should be careful not to throw the baby along with the bathwater.

For most purposes, search engines seeing you = Googlebot seeing you. I have never had a competitor past single digits in percentage of search engine referrals.

A crawler is not necessarily providing content to a classical search engine, as it can serve many purposes. Ultimately such crawlers may generate links to your website and while not bringing you too much referrals traffic, it may increase your ranking, which ultimately means more traffic from Google.

Also, if search engine referrals is all you're interested in, if Google drops you from their index, your website might as well not exist, right? As I said, be careful not to throw the baby with the bathwater. Google is not what made the web great, but a Google dependency might break it and website authors can be blamed for this.

However by using hashbang URLs, you have to keep that little snippet of JavaScript forever to maintain permalink even after the web has moved on to another solution.

It's not a future-proof solution.

Hashbangs are only a stop-gap at this point - modern browsers ship with the HTML5 History API.

Does IE8 ship with the History API?

No, IE8 is not a modern browser by most definitions.

Which definitions? I've seen definitions of "modern browser" that only exclude IE 6 and 7.

They are wrong. They use modern to mean "quite new and available to most clients" instead of "actually implementing the majority of the latest standards".

IE 9 comes close, and can be considered modern.

I'm not sure what you mean by little snippets of javascript, a link is a link. If you change why couldn't you just 301 them to your new format? It's a trivial redirect that you're keeping for the g-juice.

The part after an hashbang is not sent to the server, so you need client side code to load the content.

This is because the # in an URL is supposed to be an identifier of a element in the page, not an actual resource.

Or you could just keep a tiny snippet of Javascript that searches for old style hashbangs and redirects all of them to the "another solution" handler to decide what to do with them. It can be just 3-4 lines of JS.

The problem is that you have to keep that snippet forever if you ever use the hashbangs. That's why I'm saying it's not a future-proof solution. Not that few bytes matters, though.

Your first point is not correct. Crawlers can crawl JavaScript only websites using the Ajax crawling API: http://code.google.com/web/ajaxcrawling/docs/specification.h....

This sounds like a huge imposition to implement -- so much so that I seriously doubt anyone will bother for anything even moderately complex. Not only do you have to produce an API for the client-side javascript to call, but you also have to render static HTML for Google to index.

Reply on news.ycombinator.com