Twitter to move away from Hashbangs | Hacker News Reader

Twitter to move away from Hashbangs

storify.com

363 pointsChrisArchitect77 comments

This is fantastic news.

From the recent tweets by https://twitter.com/danwrong it looks like Twitter are moving entirely to HTML5 pushState, and leaving IE users with full page refreshes rather than continuing to serve them #! - Dan says "I'm not sure why everyone is so adverse to page refreshes these days. You can make them fast too."

Of course, Twitter are going to have to include a piece of JavaScript on the http://twitter.com/ homepage which checks for a #! and redirects the user to the corresponding page - and they'll have to keep that JavaScript there forever, since they have nearly two years worth of links that they need to avoid breaking. One of the many reasons #! is such a nasty hack.

In terms of performance, this is going to make Twitter a lot /faster/ for me - I often open Twitter profile pages in new windows (due to working on Lanyrd) and each new window has to pull in and execute a HUGE chunk of JavaScript before it will display the page. Being able to just load a regular HTML page will be much faster for me.

can't they just do a one-line .htaccess regex replace? or is it more complicated than that?

The redirection has to be done on the client-side, because the server never receives the fragment identifier as part of the URL.

Nope, because anything after the # isn't included in the HTTP GET request made to the server - so a .htaccess-level redirection won't see it. You have to execute actual JavaScript on the client to read and redirect the #! bit.

That is the fundamental problem with hashbangs: the server receives "GET / HTTP/1.1" and the client uses javascript to render the hashbang. The server has no idea a hashbang is present until after it sends javascript to run client-side.

It doesn't sound like they're switching to rendering on the backend, though. So you're most likely still going to have to execute a huge chunk of javascript before displaying the page, though hopefully it will get faster.

Dan doesn't say anything about where the rendering will happen.

Most sites these day (including Facebook, Gmail, etc) have to execute "a huge chuck of js" before displaying the page.

I have been told by some people in the know that they have come up with a system similar to pjax for certain page components, though that may have changed.

True for Gmail, though Google+ renders most of the initial page view in static HTML/CSS, then progressively enhances subsequent pages using JS and a variety of techniques, including embedded frames and pushState. There's a lot that goes into making G+ so fast, and pre-rendering a big is part of it.

By way of comparison, view the following two links in a javascript-less browser (or just view source):

  $ w3m http://twitter.com/#!/danwrong/status/171680703824662528

  $ w3m https://plus.google.com/117377434815709898403/posts/f35f8QvHab7

The first link contains nothing but boilerplate and the JS that lazily loads the real content, whereas the second contains the actual content ready for the client to render immediately (and index, if you're a crawler).

This is a good move by Twitter. I wholeheartedly endorse it.

Wouldn't they have to, to support browsers without pushState? And even if they used terrible JavaScript hackery to get around that (serve same content for every URL, have clients AJAX in content based on location.href) how would they support web crawlers? I thought the whole point of the hashbang was that you would get the original content (or an equivalent representation) by removing it from the URL, so I'd think they still support backend rendering at some level.

They could use the exact same setup they have right now.

They'd just serve the exact same (static) html for every twitter.com/username URL. I don't think it's exactly terrible Javascript hackery to look at the location.href and do an ajax call.

They'd probably support web crawlers with the same system they use now. And many web crawlers these days execute javascript anyway.

The system they have now is the #! hack, the crawlers know how to request static versions of the pages. There isn't a similar mechanism for pushState based designs, you just serve up the content pre-rendered for the initial page. So that's probably what they will do to stay indexable. It will also make the first page load snappier (probably)

I'm hoping that they plan to serve up the HTML and CSS for the basic page as quickly as possible, then load the JS that adds the pushState / interactive stuff asynchronously.

A page with a single tweet on really shouldn't need to pull down much code. The JS can all be cached by the browser (far future expires headers etc) and there are some neat tricks you can do with app cache and localStorage to make that work even better.

Indeed, one always hopes they do the right thing :P

This is what they do.

Woop, sorry I downvoted you.

Nope.

At the moment if you go to https://twitter.com/#!/twitter then it loads up the HTML for https://twitter.com/, if you're logged in it will be the tweets of those you follow. Then the javascript magic loads in @twitter's profile dynamically.

PushState will allow twitter to load in the static HTML for @twitter's profile at https://twitter.com/twitter/ and then progressively enhance it with javascript. The net gain will be that the page will be available much faster on initial page load.

Well, you're right and you're wrong. I'll be specific, since you are:

Twitter is sending down the home timeline as HTML. In fact, what most people are looking for when they go to twitter.com is their home timeline so I'm correct, this is what they are doing (at least in new-new-twitter which I think almost everyone now has). You're also correct, when you go to twitter.com/#!/n8agrin you get your timeline in the background which is then replaced with my tweets. However it wasn't always this way. At one point in time, going to twitter.com just downloaded a bunch of javascript and an HTML bootstrap. The whole site was generated by javascript. I know this because I worked on it.

Technically pushState doesn't really have anything to do with the practice of progressive enhancement or making anything "much faster". You can make pure javascript sites very fast, you can make serving straight up HTML very fast, you can progressively enhance HTML without pushState. pushState is really just a way to manipulate the history of a browser, while optionally changing the browser's current url, without a page refresh (read more here: https://developer.mozilla.org/en/DOM/Manipulating_the_browse...). You can use it to do a lot of things, one of which is to have it serve as one part of a solution for sites that behave like single-page apps without having to resort to the fragment identifier (http://en.wikipedia.org/wiki/Fragment_identifier), forever vilified as the hashbang.

IME I get my timeline flashing up in the foreground (not background!) before the actual content I want appears. Wastes a second or two.

this will certainly happen, at least in the places it will be most useful.

IIRC, Twitter brought in Steve Souders as a consultant for improving the performance of the Twitter web interface and the main recommendation he offered was to have the tweet content in the HTML page instead of loading them dynamically.

Dan says he is on the job.

https://twitter.com/#!/danwrong/status/171730032807518210

To be honest, I'd prefer maintaining a single JS redirect on the main page, rather than many server-side regular expressions trying to redirect to appropriate pages.

The actual URL for the informative blog post is this: http://danwebb.net/2011/5/28/it-is-about-the-hashbangs

The linked blog post only contains some relatively meaningless Twitter messages and the hyperlink as text, not as an actual link.

One of the things the post doesn't mention (it's sort of implicit in "going under the radar") is that with hash bangs, every request has double the round trip time to retrieve the initial data being displayed, as the server cannot know what data the client wants to retrieve. This makes a lot of nifty performance optimizations impossible.

Only inter-site links incur the double round trip (and in NewTwitter's case, serious load time); intra-site links should be faster.

"intra-site links should be faster"

Why? With proper caching, all you need is the new content. Which possibly entails a few KB of HTML boilerplate and a few milliseconds for the browser to render it, but the round trip is the lengthy part and you get that either way. The only thing hashbangs get you is a prettier transition.

No. That link is to an OLD blog post explaining why Dan Webb (and many other people) think hashbangs are bad idea.

The actual news here is that Twitter is moving away from them, which is very new and welcome news. That link is only useful for general background on the hashbang argument; it's otherwise irrelevant.

Good. A lot of developers justified doing it in their own projects because Twitter and Gawker were doing it. Now that one of the headline sites is no longer using it (and will hopefully condem it) we can file this episode to history and never speak of it again.

Edit: wouldn't it be awesome if Google (they did start this, afterall) would allow sites using hashbangs to auto-update all indexed URls

Do you know how long ago Gawker dropped them?

~6 months ago IIRC

they constantly had problems with their redesign and the last update involved removing the hashbangs

This sounds about right, if not a little earlier. Experimental work for this was done by me here: https://github.com/ianvanness/html5history

I believe there were further tweaks for additional pages that needed to be supported (whereas I had only solved, and most likely not very elegantly, the rudimentary issue) that took additional time before the hash bang removal was actually launched.

Thanks - I was estimating off the top of my head - the site was in flux for a while at the time. They also went back and forward with keeping the story sidebar at fixed positioning and it seems they settled on it not floating.

I check the site every couple of weeks because I was interested in how the redesign would work out. It seems they eventually got it right, I like it. A lot of the other blogs have home and article pages that are just too heavy and slow.

hash bangs and push state are essential for creating single page apps.

Hashbangs for state representation are just a tool, which can be used to make things better or worse for the users.

Where content is the main purpose ("websites"), they are overkill at best.

They can be useful though in functionality-first applications ("apps"), where the interface can be costly to build (too slow if you reload the whole "page" on each state change). Ideally, the app differentiates between fundamental states, represented in URLs, and transient states, represented in hashbangs.

Anyway, this is good news for Twitter, if they are really going through with it.

I believe the best implementation of this patten comes from the trance.fm website. Trance.fm gets that people come to listen to their online radio, so avoiding a page refresh is priority #1. It's very quick and responsive, too. You can listen to the music as a non-member, become a member and rate the song you're listening to, all without the music stopping. I believe it makes for a really good experience.

I'm particularly happy about this because I spent a bunch of time yesterday trying to get any of the current lightweight web browsers (Dillo and Links2; I couldn't get Arachne not to crash at startup, and I couldn't find the source to hv3) to log into either twitter.com or mobile.twitter.com, with no success at all. m.facebook.com worked, more or less, but no love from Twitter.

Apparently you can't log into Twitter without 64MiB of RAM, which is what Midori uses. Dillo and Links2 use 6MiB.

Hopefully this means they're switching to a more normal HTML interface and will become a web site instead of a DHTML application again.

Twitter's implementation of the hashbang was awful. It broke the back button and it was slow. I don't think it's a fair representation of the technique.

EDIT: And based on their implementation, I wouldn't trust anything their engineers have to say about hashbangs either.

In what sense did it break the back button? Back takes you to the previous thing you clicked correctly.

It didn't break back. It just hijacked the browser's history by adding a 2nd page load in order to inject the "!#" into the address bar. Pressing back would, as a result, act like the refresh button.

I don't follow you. Changing the functional behavior of the back button _sounds_ a lot like broken.

I'm not sure that this is the case.

Back works as expected - it takes you to the previous page you were at. Fragment identifiers don't break the back button. That's all from a user perspective's point of view, which is mostly all that matters in the end.

If you look at this from a technical standpoint, then back is a refresh of twitter.com, yes.

YES! For anyone with a high latency (huge swathes of the US are still stuck with satellite or mobile and the tech industry seems to have ignored this), twitter is a nightmare. The first pageload only pulls the empty 'framework' page, then a series of js requests pull the information. You can't walk away while it loads, either, because it will register the latency and display errors instead of content.

Good. Now if they would get rid of minified urls they would be done with their damage to the web.

I've been using the mobile version on my laptop for a few months now since the hashbang was so sluggish, but this could get me to switch back.

Can somebody explain to non-web-types why this matters, other than making the URL itself look cleaner?

The fundamental problem is that when you use hashbangs, if you want the page "/#!12345" the server gets:

    GET / HTTP/1.1

And never knows what content the client is trying to access, until the client retrieves and processes client-side javascript. Then the client can request further information from the server. The theory is that you can make a "single-page" web-app that loads all of the javascript at once. Then went state needs to be updated, you can do it all from javascript. In practice, it didn't work as well.

And to expand a little on what mbreese said, the basic, fundamental idea behind the URL is for a client to indicate to a server what resource it's trying to access. So it's totally broken by that #!.

http://isolani.co.uk/blog/javascript/BreakingTheWebWithHashB... is a great article.

It was hinted at in other comments, but when you go to http://twitter.com/#!/danwrong, what you're really doing is going to http://twitter.com/, processing a chunk of JavaScript that looks at what's after the #!, then loading up danwrong's page. This causes issues with search engines (the server has already told the crawler everything's OK, so how do you tell it the page can't be found?) and changing URLs (if you want to stop using hashbangs, you have to include an annoying piece of JavaScript on your home page forever).

So now, Twitter wants to get rid of the hashbang from its URLs because of— in part— the first reason, and it's currently dealing with the second.

Hope that explained a bit.

Thanks, that's what I didn't get: the stuff after the hashbang isn't sent directly to the server as part of the GET request but interpreted by JavaScript after the page starts to load.

Actually, twitter should not have any google indexing problem. Search engines have provided new scheme for indexing hashbang URLs with Ajax content[1]. But, maybe, not all search engines have implemented it yet.

If I had to guess, primary reason would be that server is blind as to what page needs to be loaded when you click twitter.com/#!something and an ajax request actually serves the relevant content, so it's relatively slower. Twitter's implementation was particularly slower.

[1]https://developers.google.com/webmasters/ajax-crawling/docs/...

     Search engines have provided new scheme for
     indexing hashbang URLs with Ajax content[1].

This is a myth.

First of all there is no standard and Google has no way of knowing if a website's implementation of hashbang URLs behaves in the expected manner, or not.

Then, you've got all those other issues ... with hashbangs you have no standard way of telling Google some content moved (302, 303) or is not available anymore (404).

Therefore they have to make tweaks for individual websites (akin to how Microsoft was pushing "special" fixes for various popular software running on Windows that broke on updates).

Telling people that hashbang URLs are indexed is exactly the same as telling them that Flash content is indexed ... sure it is. But unless you're too popular for Google to ignore, then get ready for a world of pain.

    This is a myth.
    First of all there is no standard and Google has no way of knowing if a website's implementation of hashbang URLs behaves in the expected manner, or not.

https://developers.google.com/webmasters/ajax-crawling/docs/...

However you are probably right on everything else.

Several issues: back button programability, URIs become uglier so then you need to do some parsing, etc.

The sample the pros and cons of hashbangs, refer to the discussion on this page: http://ask.metafilter.com/187222/Whats-wrong-with-using-hash...

Thank god. I've always felt uncomfortable that every link normal users share (I manually remove them) is technically pointing at their front page.

I especially agree with that last comment, people are so hyping over not making pages refresh for speed, they rather often trigger the opposite effect.

As far as I know, the proper way to create a modern web app to deal with location.hash fallback and pushState, replaceState etc is to use https://github.com/balupton/History.js .

Comments/Corrections are welcome.

I thought this was about twitter moving away from hashtags. For some reason I got a bit excited, may be because I find vast majority of hashtags to be annoying noise. That said, I know they serve a purpose in specific use cases and donno a better alternative.

Is it un/reasonable to consider moving hashtags out of tweets and into metadata or just separate?

Just as useful, but can be styled separately. Evolved hashtags, backwards compatible.

That would make sense to me. I always thought it was odd that you weren't allowed to attach a link as metadata, causing the rise of all the various URL shorteners.

The tweet still needs to fit into 140 characters, including any metadata you want visible to the recipient. Think SMS.

Hence, might as well make it a user-manageable item in the tweet itself.

I know that was the original idea, but I wonder how many people in the last year or two receive tweets via SMS. If I was in charge, it's a tradeoff I'd be willing to make.

The entire world is not the United States. :) "Feature" phones and spotty data service are still very common in certain regions, but SMS works almost anywhere with cell infrastructure (or via sat phones).

But then how useful are URL's in an SMS? I would trade off a slight decrease in usability for dumbphones for increased legibility for PC/smartphones.

I wonder if we don't have the technological means to send short-urls only through SMS and real links everywhere else.

Not quite true -- tweets can include in_reply_to metadata, indicating the last tweet responded to, and location metadata, indicating where the tweet was sent. And of course, the account that sent the tweet is metadata, and so is the time it was sent.

And for my money, the reason they wrap every link through t.co is because this way they have analytics on every link clicked on Twitter. That sounds pretty valuable to me.

Hashtags were/are a Twitter user creation that they do very little to support apart from making them linkable within posts so they are easily searchable from their application interfaces.

Seems a bit disingenuous saying they do very little to support it. There was a big UI change in the latest official Twitter client that made a major tab for trends and hashtags. Seems to be a core facet to me.

Yes and no. First of all, at least until recently, a hashtag wasn't actually distinguishable from the word itself. So 'ycombinator' and '#ycombinator' would both show up in a search for 'ycombinator' (and that was intentional. If anything, they were making it easier for people to ignore them (if they wanted to).

Second, the hashtags (as well as the @replies) both grew organically out of the userbase. They adopted them because the users had already adopted the hashtags. In fact, if I remember correctly, they were initially resistant to the idea until they saw how much people took to them and used them even without any official support.

The new tab seems like a glorified search - they use a hashtag for an icon, but it seems like the system itself would work pretty well for any single-word search. And with a data firehose as fast as Twitter's, you don't really want to have to do search over multiple words if you can avoid it.

Dropping hashtags would be suicide. It's one of the elements that make Twitter unique, and it's even managed to break into mainstream culture in most countries. It doesn't matter that they're full of noise, people just love to tag #somerandomstuff and see it spread (or not), gives them a feeling of belonging, the holy grail of social networks.

Us nerds, we won't care one way or the other anyway.

"PushState or bust" means there will be no support for IE version 9 or earlier, which makes me wonder if they'll take the approach of layering replaceState on top of the existing hashbangs thereby fixing the deeplinks issue.

OTOH isn't this the "modern way" to develop a web app?

HTML templates + AJAX on the client and a REST/JSON API that you can reuse for iOS apps on the backend?

I wouldn't say the re-use is specifically for iOS apps, but in general, you're right.

Even though Twitter has decided it's either "pushstate or bust" what sort of fallback for pushstate exists for those of us who care about all our users, not just the ones with good browsers?

The fallback is a normal web request and page refresh. This seems far more caring for all of their users than the high load that the hash-bang design placed on client computers.

In their case, yes, but the potential speed gain by pushing a lot to the client-side is great when properly harnessed.

How about they move away from URL shorteners too?

That totally makes sense. Time to pushState!!!

Dan says "I'm not sure why everyone is so adverse to page refreshes these days. You can make them fast too."

Rolls eyes I don't need Twitter (140 characters should be enough for any application!) telling me how "fast" their pages load. Particularly coming from the notorious fail whale.

Reply on news.ycombinator.com