Breaking the web with hash-bangs
isolani.co.uk
isolani.co.uk
The author clearly dislikes what's going on, and the post would be stronger if he simply stated the disadvantages and let others speak to the benefits rather than putting up a strawman and claiming that people implement this sort of site because it's cool.
Both of these can be achieved using progressive enhancement. There is no technical reason to include it in the page itself.
hopefully html5 pushstate will alleviate a lot of the complications with providing full fallbacks
Full (unbroken) link: http://calendar.perfplanet.com/2010/thoughts-on-performance/
As it sits, we use a full client side UI library and push all UI concerns into the UI, they communicate with the server via REST to get data and the client is responsible for rendering that data. We research our numbers extensively and the cost analysis of changing our development model to support clients that cannot support full JavaScript clients supports the conclusion "at least for us" that a progressive enhancement development model is not cost effective.
The money recouped with the shortened development cycle of Ajax clients outweighs the amount we would receive chasing some fraction of < 2% of potential clients, If we get to the point that we need to chase that 2% for growth we are a) doing great and b) would be better served segmenting that traffic to a completely independent server framework based UI that it built for those clients.
It is my belief that mixing the two development models gives you the worst of both worlds when it comes to features and maintainability.
Others may want to avoid the overhead with a full page
refresh as users navigate the site.
Avoiding a full-page REFRESH only makes sense when the current page does NOT change (e.g. doing login).Speaking about overhead, to avoid bandwidth problems related to page-switching 90% of websites out there would be better off if they just added the Cache-Control on their static assets or GZip the content sent (yes, it's kind of ironic that so many developers ignore common-sense techniques already available in the HTTP protocol).
Also, if the overhead is in the server-side processing done, then offload that processing making it async and cache the results. Why is that so hard?
For me Twitter's new interface is slower and more cluttered than the old one, without providing the benefits of the iPhone client I'm using. It's a disaster IMHO, so speculating they only did it because it's cool is not unreasonable.
Avoiding a full-page REFRESH only makes sense when the current page does NOT change (e.g. doing login
User navigates from "A" to "B" Some or even most content on the page changes. Scenario 1: New page is fetched from the server. Scenario 2: Content is fetched from the server via AJAX and rendered with a template in the browser, then injected into the current DOM.
I assert there are technical benefits to the second scenario. Whether they outweigh the disadvantages is not my argument. My argument is that there are technical benefits and that "because it's cool" is a straw man argument.
if the overhead is in the server-side processing done, then offload that processing making it async and cache the results. Why is that so hard?
Yes, and one such strategy is to offload it to the user's browser. Again, I am not arguing it is the best strategy, as I am not responsible for Gawker Media or your application.
I'm not disputing that the author has proposed a way forward or that the author's suggestions about the costs of using fragments are mistaken, I am only disputing the argument that there is no reason to use url fragments besides "coolness."
The history API in HTML5 allows you to accomplish the same thing without the hash-bang, but browser support is limited. I favour using the history API for progressive enhancement and defaulting to traditional page requests.
It's unbelievably neat to transition between pages without ever redrawing the full screen. Pity there won't be any IE9 support.
Also, you'd want Expires, not Cache control. Otherwise you may end sending dozens of request just to get 304 back.
Keeping that in mind, sending single AJAX request which delivers _only_ content seems quite beneficial compared to full refresh scenario.
That's called a domain name. www.lifehacker.com is a single gateway to all of lifehacker's blog posts. Then you use the path part of the URL to specify which post or collection of posts you want.
Twitter hasn't leveraged anything. The tweet URLs without the hash-bangs were more accessible to Google.
You want tight-control as to which user-agents can see which set of tweets - that's what the Controller is for on the server.
He suggests using real URLs - instead of "/#!5753509/hello-world-this-is-the-new-lifehacker" we would have "/5753509/hello-world-this-is-the-new-lifehacker". Sounds great, but what happens when you click to a new article, which is loaded with AJAX?
> Engineers will mutter something about preserving state within an Ajax application. And frankly, that’s a ridiculous reason for breaking URLs like that.
Is it? Let's try things all possible ways (with the assumption that content is loaded via AJAX) and see what happens. Firstly, the current state:
Address bar: "/#!5753509/hello-world-this-is-the-new-lifehacker", click on a link, now it's "/#!downloads/5753887". I can bookmark this, send the link to a friend, etc etc.
Option two is to not track this at all, after all preserving state isn't that important, right?
Address bar: "/#!5753509/hello-world-this-is-the-new-lifehacker", click a link, it doesn't change. I can't bookmark the new page, I can't send the link to a friend, my back button won't work properly.
Third option is to use real URLs as he suggested... we need to track state (otherwise we hit the same problems in option 2) so let's try that: Address bar: "/5753509/hello-world-this-is-the-new-lifehacker", click a link, it now contains "/5753509/hello-world-this-is-the-new-lifehacker#!downloads/5753887". Oops. That's even less clean.
Conclusion: If you must load content with AJAX, using URL fragments to track state is the most functional and cleanest-URL option available.
I admit my Google searching turned up no useful results about hash-banging URLs for new websites, only Google's attempt to index sites that ignored progressive enhancement and hid all content behind Ajax calls.
"Sounds great, but what happens when you click to a new article, which is loaded with AJAX?"
If you want to load the content in via Ajax, you start with a valid and direct URL to a piece of content. You can then use JavaScript on the click handler to mangle the URL into a hash-bang, and then proceed as you wish. You get all the benefits of hash-bang without the downside of uncrawlable URLs.
"now it's "/#!downloads/5753887". I can bookmark this, send the link to a friend, etc etc."
* Bookmark it to a third party service - yes (assuming they don't truncate fragment identifiers!) * That service then retrieves the content of that article (for summarising, entity recognition, extracting microformatted data). no.
The hash-bang URL is not a direct reference to the piece of content. The indirection shenanigans isn't part of HTTP/1.1 or RFC-2396. So it's not Restful for starters.
"after all preserving state isn't that important, right"
That can be done without making all links on the page non-traversable with a crawler.
"Conclusion: If you must load content with AJAX, using URL fragments to track state is the most functional and cleanest-URL option available."
a clean URL would not be an indirect reference that would need to be reformatted into a direct reference before being curled.
State can be done without mangling URLs in this way.
I wish, unfortunately this is only supported in the very latest browsers (I've literally seen one website do this [1]). You can only change the portion of the address bar following the "#" with javascript, which is why you see URLs like "/content/30485#!content/9384" in some places.
The short of it is, if you insist on using AJAX to load your primary content, and to power your primary navigation, then the way they do it is the best method at our disposal at this time.
Personally, I don't think the best method is good enough, but right now there is no better solution - imo, the only way to avoid this nastiness is not to load your content using AJAX in the first place! I don't see the benefit in throwing away half of HTTP, all of REST, breaking inter-operability and SEO, requiring javascript, and having ugly URLs, in order for... what? It's hard to see the plus side to be honest, this is categorically the Wrong Way to make a website and I wish people would stop doing it!
Okay, you are clearly missing the solution here.
This is what the html looks like: <a href="path/to/page">Page</a>
Then add a click handler to the link (using your preferred JavaScript library): $('a').click(function(e) { this.href = "#!" + $(this).get(0).href; });
So this code mangles your URL into a hashbang with JavaScript when the link is clicked. Now the rest of your code remains exactly the same, and updates the window.location.hash the same way as before - it gets the same value as before.
Except the benefit is that with no JavaScript - like a crawler - sees a working URL. And the JavaScript then mangles the URL appropriately, leaving your framework blissfully unaware of the difference.
This is progressive enhancement - using JavaScript to mangle links that are only meant to work when JavaScript is available (and in Gawker's case, make it to the browser). If you're quite confident in your JavaScript skills, you can mangle all the links on the page in one go right at DOMReady.
The key is not expecting JavaScript to be running. And when JavaScript is indeed running, then mangle links to your heart's content.
This, I believe is a good use of custom data attributes. When your JS introspects the html, it can use data here for configuration and setting values. It's messy and often impossible to use the same values you output to html in your javascript models. (For instance you might have a Number with arbitrary percision on your model, but only spit it onto the page with decimal points.)
But, yeah sure. It's a some extra work and planning we don't always have time for.
The problem with having this mix though is that someone could visit "/45382/some-content-url" with JS turned on and then click a mangled link - taking them to "/45382/some-content-url#!94812/some-other-content" which is absolutely horrible.
As far as I'm concerned, the only real solution is to not use javascript for loading pages rather than, you know, invoking page loads. I really don't see why this method is beneficial at all, there is nothing of merit in the design or functionality of Gawker in my opinion.
Well you've already confirmed that the client is running Javascript so:
1. Use JavaScript to rewrite the existing clean URLs into JS function calls (or however you want to handle that)
2. Use window.history.pushState to push the clean (canonical) URL for the page you're viewing to the browser.
I'm not sure why this isn't obvious...
http://www.asual.com/jquery/address/samples/express/portfoli...
Well, if the client isn't modern, 301 them to #!/content/456/ and proceed as you would.
Either way though, all this is terrible for a blog, or even Twitter.
I would deliver the page with plain old HTML links to other pages. On page load I'd use JavaScript to see if the new pushState[1] API is available, and if so fetch the JS required to update the page dynamically and then update the link elements to do the AJAXy thing. Seems like it would degrade gracefully in old browsers and keep the URL current in new browsers.
[1] https://developer.mozilla.org/en/DOM/Manipulating_the_browse...
edit: is sjs382 my doppelganger?! (or vice versa)
Haha, not at all. Just my old university ID since "sjs" was taken here ;)
Personally I think it's a little early for pushState, and degrading to #! is really ugly, so for now I would say just use normal page loads like everyone else unless you have a really good reason (good reason: GMail. bad reason: glorified blog.)
[1] http://tkyk.github.com/jquery-history-plugin/ [2] http://benalman.com/projects/jquery-bbq-plugin/
It's just nuts that I have to burn so much CPU just to read some text.
Furthermore, I've never been annoyed at page reloads while reading a blog. Now their shitty transition crap?... There is nothing "lower latency" here.
And the same way I would rather create JS-driven sites. It allows me to be more creative with the presentation, while staying in the comfort zone of HTML/CSS/JS. Offloading work from my servers to the computers (that, based on my experience, are near-idle while browsing) of end users is a nice benefit.
Oh, and I would disagree about the 'There is nothing "lower latency" here' statement. I visited lifehacker.com out of curiosity, and noticed something: with JS navigation, I could scan headlines in the sidebar while the article was loading, something I would not be able to do while a 'traditional' page was rendering.
Of course, it's not a general rule. There are projects where you support IE6/7, and then there are projects where you don't give a damn whether it works for anything but edge versions. Same with using JS as presentation layer.
...Wha? You can make traditional pages that display progressively just fine. In fact that's the default mode of operation for HTML and was supported as far back as Netscape 1.0. What's it with people, why do they keep forgetting about sensible architectural decisions just because they were made long ago? Or am I just getting old?
You know what is otherwise nearly perfectly analogous to telegraphs? Texting. That stuff is popular as fuck.
Who cares? Only you. Gawker cares about lower latency = more pageviews per visit = more $$$ from ads.
And does anyone besides me find it ironic that the term 'thin client' is sometimes applied to this hairball over ever-increasing complexity we call the web browser?
over ever-increasing complexity we call the web browser?
Much of the complexity at least from a development standpoint is when people try to sprinkle in Ajax. They are two very distinct development patterns and mixing them creates a maintenance monster. If one is providing pages they are better off, grabbing a CMS and delivering pages, if one is building an app then one is better off throwing away JSP, ASP or PHP and writing all of the UI logic in client side JS that communicates to the server via REST and gets data that the UI is then responsible for displaying. Anything in between those two becomes a maintenance monster.
And if you think that doesn't sound like a good reason, you'd be right.
I.e.:
1. You visit "/1234/spam", and get served full HTML page with some (optional) JavaScript for progressive enhancement.
2. You click on "/4321/ham" link, but JavaScript hooks it up and replaces URI with "/#!4321/ham" (if your browser don't support HTML5 history API, of course). Yes, there is one full-page reload.
2.1. (Alternative) Or - even better - you can be redirected to "/#!1234/spam" on step 1, so you won't notice the glitch on your first click.
3. You continue to navigate with AJAX, now without reloads. You can bookmark pages and so on (and if you somehow happen to lose JavaScript support you could just remove "#!" to get a valid URI).
Very simplified implementation cost:
1. `$("a[href]").attr("href", function() { return this.href.replace(/^\/(?!#)/, "/#!"); });` on every page
2. `if (location.pathname === "/" and location.hash.match(/^\/#!/) { $("#content").load(location.hash.replace(/^#!/, "/") + "?format=content-only"); ... }` on / page.
3. Ability to return undecorated ("article-only") content on `/1234/spam?format=content-only` requests.
The Gawker family of sites failed at doing all of this.
Seems like this should be made a JS library in the meantime. (Hint, hint. =])
With this implementation, when the link is clicked the appropriate content is served through an Ajax call, and the crawlers are able to index the content.
You can see it in action here http://lynkly.com
good one @hybrid11
http://stackoverflow.com/questions/3009380/whats-the-shebang...
The solution with "mangled" URL seems to force a reload on every new request to lifehacker as well. Just like what Facebook did with the hash-reload.
I think what the author of this diatribe fails to understand is that the separation between the part of a site that uses client side navigation, vs the part that uses sever side navigation is an architectural design choice - not an aesthetic choice of how a developer wants their URLs to look.
Client side navigation has performance benefits, ability to use web apps offline, etc. These are all valid reasons to choose to use it. It may not be ideal that the end user is exposed to that detail by breaking the URL up between the "server part" and "client part" separated by the # character. But that's where we are today in browser design.
I don't buy the argument that the # scheme makes a web site more fragile. You can break the server side of your app just as easily as you can break the client side.
1) How many of our users will be affected? 2) How much harder (i.e. how much longer) will it be to do it right? 3) Don't we depend on JS to inject ads anyway? Ads are the whole reason anybody's paying for this...
When you truthfully answer #1, if #2 is more than about 5 minutes, nobody's going to budget for it.
Overall, the hash-as-URL argument is hampered by the fact that Gawker is a prime example of going single page just because you can. Other than fancy page->page transitions, I don't see what their new setup buys them.
On the other hand, I'm impressed how you just ignored TrollBait's insult and just replied in an absolutely normal manner.
Now worrying about having to build your templates twice -- once in your back end language, and then in JavaScript -- is a valid concern. Code duplication is never good. However, mustache is supported by many many languages (http://mustache.github.com/) which means that you really can build your templates just once and call out to very similar code to populate them with data.
Yes, you would need to run tests against the site with and without JavaScript (or CSS or XHR or any other feature that you were using to progressively enhance your page, but how far you go is really up to you), but if that's automated, it isn't much effort on your part.
I'll be the first to admit that it's fine to cheat in the code to give your users a better experience, but I don't think it's a good idea purely to take development short cuts or cut costs. If our roads were built that way (and in India they are), you'd end up with potholes every six months (and in India we do).
Typical crap : you can't access nvidia.com anymore with lynx/links. When your goddam nvidia proprietary driver doesn't play well with your kernel upgrade, you have no way anymore to download a new one without running X11, though it used to work not too long ago.
We as web developers just spent the last 10 years as slaves to IE6 and saying "oh how I wish developers would/could develop for standards, not just with platform X in mind". And here we are again, setting ourselves up for an even worse situation than the IE6 problems. We have sites designed with only (iphone | IE6 | javascript-capable) in mind. The reason of "it's too much work" is the same answers given time and time again to the sites only designed for platform X, but that's not a good reason to do something when the platform you're delivering it on is by definition a network of variable capability platforms.
It's fine if we want to try to push the state of the art of rich internet apps, but at what point do we stand back and realize that we're not building a website (a collection of HTML documents on the world wide web) but rather delivering a javascript executable to our user that just happens to be renderable in most modern web browsers?
I don't mean to single you out, it's something that we all having to deal with, but is there anyone at the standards organizations that are listening to the pulse of the new web? If people want to deliver applications to users via a URI, why do we have to include all of the extra baggage of HTTP/HTML/CSS/Javascript?
If we as the artists of the web are going to break the implied contract of what the WWW is, we should at least be honest with ourselves and work toward a real solution rather than trying to staple on yet another update to HTML to try its hardest to pretend to be its big brother Mr. Desktop App.
The fragment (part after the hash), in general, is used to denote a specific state (or "page") in a single page ajax site.
[1] http://code.google.com/web/ajaxcrawling/docs/getting-started...
The problems I see is that this would require both server and browser support, and that leaving persistent connections open for minutes could be problematic.
Both:
http://www.isolani.co.uk/blog/javascript/BreakingTheWebWithH...
and:
http://www.isolani.co.uk/blog/javascript/breakingthewebwithh...
render the same content, and <link rel="canonical"> does not exist in source code of that page.
You're almost always dependent on perfect code to keep your site running, be it server side or client side. If code breaks on the server side you're just as screwed.
That third party dependency is a significant difference.