Googlebot's recent improvements might revolutionize web development

How strong of an advantage that will be in the long run is uncertain, I would rather see a web that ships pages with actual content in them than empty containers for a variety of reasons (most of which have to do with accessibility and the fact that not all clients are browsers or even capable of running javascript).

This 'new web' is going off in a direction that is harmful, coupled with the mobile app walled gardens it is turning back the clock in a hurry.

I'm fairly sure this is not the web that Tim Berners-Lee envisioned.

1. Google still needs a URL-addressable "PAGE" to which it can send Users.

2. This "PAGE" needs to be find-able via LINKS (javascript or HTML) and it needs to exist within a sensible hierarchy of a SITE.

3. This "PAGE" needs to have unique and significant content visible immediately to the user, and on a single topic, and it needs to be sufficiently different from other pages on the site so as not to be discarded as duplicate content.

Create web pages, rank as a web page.

This is a band-aid by Google. Developers created inaccessible websites (JS-only, no HTML fallback) and Google still wanted to give those sites a chance to be in the index. Like when Google made it possible to index text inside .swf movies. This did not mean that flash sites suddenly ranked alongside accessible websites. No, it only meant that you could now find content with a very targeted search query.

Don't think you are gaining any SEO-benefit from one-page JS-only applications, just because Google made it possible for you to start ranking.

And don't forget your responsibility as a web developer to create accessible content. Forgetting progressive enhancement, fallbacks, a noscript explanation for why you need JS, ARIA is devolution. If Google can index your site, but a blind user has a problem with your bouncy Ajax widget, then you failed catering to all your users. If you lazily let Google repair your mistakes, then soon you will be a Google-only website.

If you love your users, give them HTML and let the Javascript enhance it.

Projects like Facebook's React ( http://facebook.github.io/react/docs/top-level-api.html#reac... ) and Rendr (https://github.com/rendrjs/rendr/) let you use server rendering as well as the single page technologies on the client side.

I just launched a website. It's a weekly periodical with political analysis, word-count on articles 1500-6000. It needs to carve up the content in a few different ways (categories, issue numbers etc), decorate an article with links to other relevant content, and provide a nice CMS for non-tech people to use. So it's on Django, with the regulation sprinkling of JQuery. (If it were only techies updating it, you could probably do it with a static site generator...)

To me, the idea that you'd try and force something that is plainly a big collection of pages into a 'single page' is just philosophically bizarre, like printing Moby Dick on a square mile of paper, using some amazing origami skills to present it to the reader, all in order to save a bit of effort at the paper mill.

The googlebot business is one aspect of a bigger issue, which is that a website needs to be consumable by a host of different clients. I don't see how you can do the SPA thing without making major assumptions about those clients.

Sometimes, of course, those assumptions can be justified - it depends on the job. And Angular etc are enormously fun to play with, and handled well can enable a great UX for certain jobs. But I don't think it's 'the future'. It's another tool in the box.

Relevant here, a nice talk by John Alsopp from Full Frontal 2012:

https://www.youtube.com/watch?v=KTqIGKmCqd0

EDIT: clarification

A simple guide can be found here: https://developers.google.com/webmasters/ajax-crawling/. Although I suspect it needs to be updated since its from 2012.

If you create an application, make sure it alters the URL when applicable. For simple apps, the following repos will be useful:

The old way, that still works: https://github.com/asual/jquery-address

The better way, preferred: https://github.com/browserstate/history.js or https://github.com/defunkt/jquery-pjax.... not sure which is better to be honest. Feel free to chime in.

If your users (I'm talking humans, not bots) have to download a mountain of JavaScript and execute it before seeing any content, your site is slower than it could be for everyone. We should stop saying that "single page apps", i.e. sites rendered in JavaScript in a browser, are bad because they can't be scraped by a bot. They are bad for EVERYONE who wants to view the site because of the network and CPU time it takes to download the assets and render the site in the browser.

There is some strange logic there.

I find the cheerleading for single page websites disconcerting and the proposed benefits unconvincing. Why should this be the default way to build websites? A few desultory upsides are presented without a full consideration of the multiple downsides to client-side development.

The biggest advantage of thick-client architecture is sending less data to the client and, if you like using javascript, writing everything in js, but there are multiple downsides compared to more traditional thin-client websites - load times which depend more on client capabilities (hugely variable and out of your control) than servers, dependence on js on the client, loading pages while your content is placed in the dom by js, forcing everyone to write in js instead of switching language on the server whenever they like, ignoring the simple document model of html served at predictable URIs, which has served the web so well and means you can use dynamic or static documents, full documents can be cached for very quick serving and by intermediaries, etc, etc. Of course some of these can be overcome, but there are serious obstacles, and the advantages are meagre to non-existent unless you enjoy javascript and feel its the only language you'll ever need.

For someone who doesn't like working in js, and/or doesn't have a huge amount of logic already in js (many websites work just fine with some limited ajax), trying to force every website into the procrustean bed of client-side development is not an appealing prospect. I can see why it appeals to those who have already invested in js frameworks, but predictions of its future dominance on the web, like predictions that eworld, activex or mobile would replace the web, are overblown.

I suspect the birth and death of Javascript will be a footnote in the history of the web, rather than taking it over as this article suggests. If anything we should be looking to replace our dependence on js, not making it mandatory.

I really, really dislike this approache of doing website, it make the code and the design hard to understand and cost a lot of problem when comes the time to debug or made major changes.

There are no magic when using javascript, it will slow down the client, and manipulating DOM is very slow. Doing things server side cost nothing compared to javascript. Remember that loading a webpage is very fast when you have only a CSS and html code, because it very easy to put it in cache and doing some pretty nice optimizations on it.

With frameworks, making "webapp" become a huge nigthmare, things becomes overly complex and bloated, the request as to pass trough a lot of layers before end up somewhere, and the developpement is not faster than have a custom code. Good framework don't make good programmer.

When the article say "put the CSS inside a <style> tag on the page - and the JS inside a <script> tag", it's just horrible, fuck it.

At any case, if you want to have a solution to this SEO problem now, I created SnapSearch (https://snapsearch.io)

There is a lot more to it. I am pretty well known as an SEO, and while I would love this to be true it isn't.

Google's improvements to GoogleBot are mostly targeting spam, and obfuscation of content. The ideas is not to discover content as much as it is to avoid having content hidden.

Previously you could have a webpage that appeared to be about Puppies, but then used any number o dynamic methods to instead show naked cam girls. Google worked to fix this.

Google is now doing some indexing of named anchors, and this allows for linking to a page with in a page as it were. But that is a Long ways from building indexable single page applications.

-Brandon Wirtz SEO (formerly Greatest Living American) http://www.blackwaterops.com

1. The state of JavaScript-only application development is still nascent. The number of JS-only sites I see that are buggy, don't use PushState correctly, or have other shortcomings is growing faster than the overall trend. Not that it can't be done well, but if your JavaScript-only "app" is really just a standard website, you might want to re-think your approach.

2. There has to be a better way. There are distinct benefits to approaches in caching content and providing feedback, but JavaScript seems to be a kluge-y approach. It reminds me of frames back in the day. Some of this is browser support; some of this is lack of standardization; some is perhaps a missing piece of the HTML spec; etc.

A single page app isn't JS-heavy by definition, and a "normal" page (with HTML generated on the server) can easily be JS-heavy. It all depends on how you program it. Just keep in mind that single page apps don't necessarily need to use heavy frontend frameworks such as Knockout, Ember or AngularJS.

As far as I can see, throughout this thread the performance benefits of single page apps are touted as being fantastic, making it worthwhile to use the new technology etc.

When has performing operations efficiently ever been the domain of the web? Websites in my experience have the worst performance of almost any software I use! I've seen developers cite 200ms or longer to load a page as being a good benchmark - that seems pretty awful to me.

If getting this tiny performance improvement (which often results in poorer performance on the first load (not ideal for many)) is so critical, why do the same developers not invest in writing more performant server apps? Yes, often the database is a bottleneck, but these problems can in general be worked around (either by use of faster queries or caching etc).

Why attempt to get a small performance benefit by saving 30-odd kB of HTML on each page load (static and so essentially free for the server), when one could get a much larger performance benefit by optimising the backend?

Almost all serious sites will still see their page load being limited by the time it takes to produce the page. It's possible to write really fast websites (try http://forum.dlang.org/) but no one seems to do it :(

If anything but almost all of your website is static, you won't be saving all that much time.

The reload via JS fails silently, when you are on a bad Internet connection, like I am currently here in Nepal. You can not simply do a reload, like with a simple HTML page.

For example, Facebook is here unusable, because of the same issue. It works only, when you request the mobile site of Facebook in the browser.

So please, get rid of that damn JS, if you care about your user base and usability.

BTW: that problem also happens on bad hotel Wifi in the US.

Either you have a highly interaction-heavy web-app, where it makes sense to execute most of the code on the client and deliver the content as JSON or you have a content-heavy website, where it makes sense to deliver cached content to the client.

There are some apps in between, which are highly interactive and content heavy like web versions of social apps. For them additionally the question arises, if they want to be crawled by Google or Google wants to index their content.

To profit from this, you need an app, which content the users search for and interact with several times after they have found it. So i guess, "revolutionize" seems a bit much to me.

We shouldn't let one company, google, dictate how the web works, simply because of their proprietary technological innovation.