How SoundCloud built its new single-page main website using Backbone.js | Hacker News Reader

How SoundCloud built its new single-page main website using Backbone.js

backstage.soundcloud.com

198 pointssteren65 comments

This is a very interesting read with a lot of interesting topics being discussed.

On "Views as components"

> Each view can include other ‘sub’ views, which can themselves include subviews and so on.

This is something that is missing from Backbone and it's something that always comes up, even when doing something as simple as a Todo list. In the Backbone Todo list example[1] you there are methods on the "parent view" (eg, the List view or the App view) to addOne and addAll (rendering the Item views).

So the concept of "sub views" is definitely needed but what I'd like to know, from SoundCloud's perspective, what were the arguments for/against implementing this functionality from within the templates and not from some place else, eg. the parent view's or the sub view's constructor (as a parent/child attribute) or initialize method.

> Each view is responsible for its own setup, events, data, and clean up.

Having already implemented the "sub views" concept it seems perfectly logical to allow a parent view to inherit DOM events from its children. Considering the Todo list example, and without using Backbone, how would you implement DOM events on the Todo items? You wouldn't bind events on each item's DOM element - that's just crazy. You would bind the events on the list (just once, for all items) and find a way to know on which item the event was triggered. In other words, you would use $('parent').on('event', 'child' ... ) and not $('child').bind.

It seems that with Backbone people have forgotten this practice, in a way similar to how people started using inefficient DOM selectors when jQuery came along.

So, when implementing the "sub view"/"parent view" concept, what would make perfect sense is to specify if the sub view's DOM events should be bounded on the parent view's DOM element. What I'd like to know is if SoundCloud has considered doing this and if yes why they have decided against it.

[1] http://documentcloud.github.com/backbone/examples/todos/todo...

> It seems that with Backbone people have forgotten this practice

To be fair, Backbone's events hash idiom is implemented using event delegation straight out of the box. It's not forgotten at all.

Although to your point: since Backbone offers no OOTB mechanism for sub-views, it offers no OOTB support for event delegation on a parent view. I just wanted to clarify where that line is drawn wrt event delegation being "forgotten".

> I just wanted to clarify where that line is drawn wrt event delegation being "forgotten".

Yes, I should have been more clear on that. I mentioned that because SC says that they may have a view (DOM element) that contains a single button (or whatever). This means they bind events on each one of those buttons, even if they have 100 of those at any given time on the DOM. Maybe they have some other form of event delegation but they haven't mentioned it.

Hi Rudas,

>what were the arguments for/against implementing this functionality from within the templates and not from some place else

It is still quite possible for parent views to construct subviews and insert them into its own DOM at any time, but due to the nature of our views, many of them are UI components, and so it made sense to be able to define both the subview as well as its position at the same time. It makes writing a view with subviews very easy. You simply use the Handlebars helper exactly where you want that view and the rest is taken care of for you.

>You would bind the events on the list (just once, for all items) and find a way to know on which item the event was triggered.

Yes, that's a very common approach to handing DOM events, but it ran against our belief that views should be independent. If a view can only work in a particular situation (eg: nested inside a list which handles its events), then that creates hard dependencies between those two views and you can quickly get into a mess. Event delegation is definitely used in Next, but only at a per-view level. It is a sacrifice (I would posit that it's a minor sacrifice), but the benefits are highly independent and reusable views. We do take advantage of this, too: for example, if you look at the waveform in the player, and the miniature waveform in the header which shows your currently-playing sound (visible in the screenshot), that is the exact same view. Because they handle everything themselves, there was absolute minimum work needed to add that feature.

Thanks for the feedback and the interest, Nick.

Note that I'm just talking about our particular project: YMMV, and using event delegation in higher level views for your own project might provide bigger wins in terms of performance or maintainability.

> It makes writing a view with subviews very easy. You simply use the Handlebars helper exactly where you want that view and the rest is taken care of for you.

It does indeed look very simple and easy but what I would be concerned about is that you have logic in your views that perhaps shouldn't be there. Your views are now dependent on the template language/parser you are using. Have you tried any other approaches that didn't work so well/easy before coming to this solution?

> If a view can only work in a particular situation (eg: nested inside a list which handles its events), then that creates hard dependencies between those two views and you can quickly get into a mess.

Agree. But how about having per view instance dependency instead of per view constructor. For example,

    <div class="listenNetwork__creator">
        {{view "views/user/user-badge" resource_id=user.id parent_view=this }}
    </div>

So your user-badge view can bind its events on the parent_view.el (if available), otherwise on itself. Backbone does this in a way on its delegateEvents method. If there is no selector specified in your event it uses bind on this.$el, otherwise it uses this.$el.delegate.

Thanks

That's not a bad idea actually -- I might look into that. With only one subview, it ends up with the same amount of handlers, so the benefit would only be seen in list-like views with repeated subviews, right?

Exactly, like when you are rendering a list of (sounds|users|comments|whatever).

    > This is something that is missing from Backbone 
    > and it's something that always comes up

... not so much missing as agnostic. You're quite right that subviews always exist in complex applications, but they're best handled in different ways depending on your templating / HTML-generation library of choice, and your app's architecture. Backbone doesn't want to force you to necessarily have a complete static hierarchy of View objects from the root DOM element down to the bottom ... many applications don't need or benefit from it. That said, if you've got a piece of functionality that you think would help with subviews in most Backbone projects, it would make for a great ticket or pull request.

    > You would bind the events on the list (just once, 
    > for all items) and find a way to know on which 
    > item the event was triggered.

Yep, and Backbone encourages that kind of delegation by default at the view level ... mostly for reasons of statelessness rather than performance. It's awfully nice to know that all of your events are always bound and ready on any given view, regardless of whether or not that view has any HTML yet, or has even been inserted into the DOM yet.

That said, binding to individual elements isn't problematic unless you're truly doing too much of it. 20 songs each with their own bound callbacks would be fine, but 2,000 songs wouldn't be. When to delegate for performance follows the same reasoning as it would in a plain 'ol jQuery app, and is touched on briefly in the FAQ: http://backbonejs.org/#FAQ-tim-toady

For what it's worth, here's an example of a Backbone view that's using delegation and a single event listener as a callback for all of the individual blocks in the charts: http://blog.documentcloud.org/blog/2011/10/entity-charts/

I'm excited about this as I am a heavy SoundCloud user but I can't help but feel this post is a bit premature. The existing state of the 'next' site is really early.. there are many bugs, most features are not implemented, and the sound detail page is almost completely broken.

I'd must rather see these kinds of posts after a big rewrite has proven the new technologies were the right choice!

You're quite right -- Next is nowhere near complete, and still very much a beta. We're working on the "release early, iterate often" approach. (See http://www.codinghorror.com/blog/2009/12/version-1-sucks-but... http://successfulsoftware.net/2007/08/07/if-you-arent-embarr...)

The main reason is quite obviously for feedback. SoundCloud is a community-driven site, and we want to get really early feedback about features, design, etc. Another part of the feedback is exactly what's happening here with sharing our techniques and seeing the results. Hopefully the outcome of this blog post is that some other people will learn something, but also that some people will point out things so that we ourselves can learn.

> there are many bugs, most features are not implemented,

We're definitely aware of the features not yet implemented, but if you find bugs (or are really missing a particular feature), please do use the feedback form to let us know!

Cheers, Nick

I've been using the feedback feature early and often (nice job putting it so front and center also!)

I think the new design and architecture is coming out great and I don't mean to discourage experience sharing. If anything, better way to have said would be:

I hope that you will post a few follow-ups to this as you get closer to release what/if anything has changed in your stack, approach, and gotchas found building this complex of a single page/backbone app.

Thanks!

Definitely. There already have been some changes since I wrote the post this week, but they'll have to wait until the next exciting instalment. :)

I think when you start reimplementing garbage collection in your application, you are probably going down the wrong path and should think about a better way to do things.

Any custom built caching layer is normally going to have to handle this situation.

What's your evidence of this? Have you built a large Backbone application? As it stands, your comment comes across as snide and indefensible.

http://en.wikipedia.org/wiki/Abstraction_inversion

It's a little bit like this anti-pattern.

Backbone.js is minimal on purpose so that you may implement these types of optimizations. That's why there's many ways to do things in Backbone.js but overall makes the application more efficient.

You know, we solved persistent background sound with hidden frames and MIDI tracks back in the 1990s. :)

Jokes aside, I wonder how on-going development and maintenance is for a "single file" website...

The source is in sanely separated and named modules in individual files which are concatentated into one for production. We actually concatenate into 4 different files according to how often the source changes (to work best with caching), but yeah, the concept is the same.

Google Closure and require.js take care of this I believe.

How this goes against the recent Twitter move to return rendering to the server? How to find the right balance of client and server functionality?

I've found that the best solution is to _not_ make your website "just another client" of your API. On the initial page load, render the page normally using your server-side templating. Then have your JavaScript introspect the existing DOM and enhance it into a very interactive close to single-page app, using client-side rendering from that point forward. Yes, there's code duplication with templates, and you have to deal with certain peculiarities like implementing a user event queue between the time the page loads and the JavaScript is done hooking into everything so you don't lose user interaction. But those are pains I'm willing to deal with.

That's a similar technique to what I'm doing; I use language-agnostic templates (mustache) so that I can do a full server side render, and then my Backbone app initializes with bootstrapped data. This way, we can progressively enhance the site and have a quick page load and at the same time build a single page Backbone app.

Do you share the same templates between server and backbone? I'm trying to escape code duplication as much as I can.

Do anyone knows any pattern to have the server templates sent to backbone avoiding duplication?

I've been using hogan.js to compile mustache templates to JavaScript functions on the server-side and write them to a js file that is then served to the browser:

  Templates={}
  Templates['templatename']=function(){/*compiled template*/}

They can then be referenced quite easily in your backbone render methods.

Hi Carlos,

It was definitely something we thought about, and even discussed with the devs from Twitter. Twitter has a very different use-case to SoundCloud. When you follow a link to Twitter, it's usually to read a single tweet (or maybe a handful), and that's it. SoundCloud is visited by someone who is already willing to invest at least a couple of minutes to listen to a tune, and is much more likely to explore the site. Therefore, the value of making further navigation of the site fast (via client-side rendering, etc) is weighted differently at SoundCloud than at Twitter.

Yes, I don't think instant loading is as high a priority for soundcloud as it is for twitter. Twitter shouldn't be a SPA I don't think, or at least not initially.

It's been a long time coming, the dashboard has been slow as shit both to download and render for a long time now.

I've been using the desktop client exclusively because the website is often so painful on my 2008 macbook, but it only shows new tracks, not all the other interactions that show up on the dashboard, which means I'm missing out on a lot of good tunes.

And advertising! Just kidding, but really some companies make the website fully featured while apps and other things don't get the updates because those are peripheries for converted users like yourself, and many models require ad based revenue that doesn't translate well to client apps or mobile platforms.

There aren't any ads on SC though. They make their money from premium accounts, of which they apparently sell plenty. Their business model is excellent all round IMO.

(You probably knew that but I thought it should be pointed out.)

> Duplication of module dependencies is also tedious and error-prone.

You can combine these with RequireJS as well. Instead of:

  define(['a', 'b', 'c'], function () {
    var a = require('a');
    var b = require('b');
    var c = require('c');
    ...
  });

You can instead just do:

  define(['a', 'b', 'c'], function (a, b, c) {
    ...
  });

Or:

  define(function(require) {
     var a = require('a');
     var b = require('b');
     var c = require('c');
     ...
  });

This approach does actually address a lot of the complaints:

- Boilerplate is greatly reduced (though still present) - Way less error-prone since you're not relying on keeping two separate lists of dependencies in same the order

But, it still does require rewriting before use (to add the module name), so I don't see a huge win here either way.

I'll also point out that this "sugared" syntax was added to requirejs after we started developing. If it were there at the start, perhaps we would have used that instead.

So when does it make sense to make a single page web application? Are there any guidelines you folks like to follow?

When there is music playing for a start!

    Wikipedia                               Gmail
       <------------------------------------->
    Traditional                              SPA

Those two I'm pretty sure what I would use, but the other applications in between those extremes I'm not that sure...

I wouldn't mind at all if Gmail was a traditional multipage app that works without JavaScript.

Google Maps is probably a much better example.

Gmail does work without javascript, it has long been the example I give of competent web authoring. As for when it makes sense, I suggest this guideline: if what you're building is to be a worthwhile contribution to the World-Wide Web, it should have a mode with server-rendered semantic markup behind URLs which are stable forever. Client-side rendering is an enhancement to build after that works well. Your content is more important than the custom behavior you're adding to the the browser the user already chose.

This is exactly the approach I'm doing. I'm writing the app entirely server based, just a couple of ajax do avoid loading unnecessary data at some time.

The time I finish the app, I will tweak and optimize loading either by AJAX or implement something like backbone views.

Yeah you should try the HTML version of Gmail, it sucks.

Whenever you can. It allows you to animate transitions during navigation, and to give your website more of a dynamic application feel instead of a static site. If done well, you encourage deeper interaction, as navigation feels more intuitive with smooth loading performance, without a jarring page reload, and with animations indicating what's going on.

Well, for a start, when you've got a fully developed API. We recently built a project that used a Backbone SPA for _some_ views, but was primarily server side.

Now we're building a mobile version, and having built the API in the first phase, we've got enough to go on to do it all via Backbone.

Additionally, when developing for mobile you're much more aware of bandwidth restrictions, so the ability to load only what you need becomes useful. When you're flicking through a bunch of pages that use the same view but with different data, an SPA lets you load only what you need.

What support does SoundCloud's new single-page interface have for IE8 and IE9? Does the new interface use the HTML5 History API to enable state/page transitions? If so, what fallback technique was used for non-HTML5 browsers? Do these browsers get a single-page interface enabled via URL fragment identifiers (#'s), or do they fall back to a multi-page interface where page transitions are handled as they traditionally have been on the web?

Is there any data on how much SoundCloud's transition to a single-page interface decreased latency during navigation relative to the traditional page-loading model?

Apologies if these questions have obvious answers. I tried to sign up for the beta but the party was full, and I'm not familiar with SoundCloud's interface.

>What support does SoundCloud's new single-page interface have for IE8 and IE9?

For IE9: some. Backbone automatically detects pushState availability and fallsback to a hashbang system allowing the SPA to work.

For IE <= 8: none.

> Is there any data on how much SoundCloud's transition to a single-page interface decreased latency during navigation relative to the traditional page-loading model?

Not yet, but that would definitely be a metric we'll be collecting. We're still working very hard on increasing the performance, so it'd be a moving value right now.

> I tried to sign up for the beta but the party was full

No worries -- you're in the queue and we're gradually expanding the rollout, so you'll get an email soon. For everyone else, you can join the beta by signing in at http://next.soundcloud.com

I'm glad he covers the commonjs / amd issue. We use a commonjs implementation as well, but it feels like more and more libraries are going to amd. Like soundcloud, the amd approach just isn't ideal when you get to a certain number of components. Seeing that they convert to amd on the fly, and use almond for production is really interesting.

So they used Ember's templating language and did a crappy, more verbose reimplementation of Ember's subviews? Why did they choose Backbone over Ember, exactly?

Can I propose a corrolary to Greenspun's Tenth Rule?

"Any sufficiently complicated Backbone program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Ember.js"

Nice corollary. Sounds about as accurate as Greenspun's Tenth as well. In good fun, mind if I propose one back 'atcha?

"Inside of every Ember.js application is a leaner, meaner, far faster, built-with-less-headaches Backbone.js app trying to get out."

I love the idea of the Handlebars 'view' helper.

I'm not sure if "insertion of subviews" only happens once or on every render()... ideally only the subviews have to be rerendered - not the parent view which is "heavy".

yeah, that was something we struggled with when designing this architecture. The problem is that you don't necessarily know which subviews can be held onto and which are no longer needed, so the only way to generically handle the situation is to destroy all subviews when a parent view needs to be rerendered. This sound horrible and inefficient, I know, but our way to deal with it has been to be very careful about what triggers a rerender.

Views state exactly which attributes on their model should trigger the rerender when they change, and no others. If you have a very high level view which has a sizeable tree of subviews underneath it, then its subviews will probably be the ones actually displaying data and that it will essentially just be a 'composite' view and will never rerender.

Views which bind to collections (eg: Lists) have special logic for adding and removing subviews without rerendering all of them.

i wonder whether "letting go" describes all of their memory management concerns

i found javascripts lack of weak references extremely problematic when trying to develop a larger backbone app

you also have to somehow manage the lifecycle of your views, cleaning up after their event bindings and such. See this stackoverflow question[1] for more about this.

[1]: http://stackoverflow.com/questions/7379263/disposing-of-view...

Thoughtbot wrote a lib that they bundled up as a gem that does this for you called Backbone Support. It's an easy way to get started, but you'll likely want to tweak it later :)

https://github.com/thoughtbot/backbone-support

I always get nervous when I see tons of JavaScript. Frankly it's hard enough to deliver a consistent experience in the browser just from a style and layout perspective.

I haven't dug into how it's done, but I feel like JavaScript apps are harder to test and verify. It's not like running unit tests on your server code. To truly verify you need to test the entire application in each supported browser.

Someday soon Javascript will have WeakMaps. They're already in Firefox.

http://wiki.ecmascript.org/doku.php?id=harmony:weak_maps

I would be very concerned about performance with a SPA built like this.

Do you have any benchmarks of Next SoundCloud's performance compared to the existing SoundCloud?

I don't have any numbers, but I switched back to the old version because "next" was so slow.

Performance is definitely the biggest problem with fat client applications. As Twitter found out, by crowdsourcing your hardware, you lose a lot of control of the user experience. If you have a 'traditional' web application and it's running slow, it's really easy to by a couple of new servers and the problem is fixed. We can't buy everyone a new computer (sorry!) and force them to use a recent version of Chrome, so it's a challenge.

In the places where we can control the performance, we do that very carefully: CDN loading of all assets, intelligent caching techniques, and of course, finding and removing bottlenecks in the code.

@jarcoal, would you mind letting me know some of your details which might affect performance (which country you're in, what browser, what speed is your computer)? Email me directly at fisher at soundcloud if you'd prefer not to share here. Thanks.

The new home needs a search form. I always go to your home page just to discover new music.

Search is in the header. Is that not what you are looking for?

And then you use a popup for the signup box?

Another blog that doesn't link to their actual site.

The link to the actual site (webengage.com) is in the header nagivation on the right with an arrow pointing at it

What? There's one link, the "SoundCloud" logo and it doesn't take you to SoundCloud's website, it takes you to the root of their/this blog.

I don't see webengage.com or a link to it anywhere on this link. Did you post on the wrong link by chance?

edit: There are also no arrows and the text 'webengage' doesn't appear in the blogpost page source...

just saw your reply, and it's not been long enough where I completely forgot the context of this, so I'm just going to apologize :) I have no idea what happened here other than a brief lapse in sanity most likely.

Ember.js FTW !

Reply on news.ycombinator.com