Stop sending template engines to the browser. How to make 6-10x faster templates | Hacker News Reader

Stop sending template engines to the browser. How to make 6-10x faster templates

andyet.net

152 pointsadambrault87 comments

Dust.js [1] has precompile templates and LinkedIn showed how much faster it is than iCanHaz [2].

[2] http://engineering.linkedin.com/frontend/client-side-templat...

dust looks neat, for sure.

>Thinking about this leaves me asking: why don't we just send the javascript template function to the client instead of doing all the template parsing/compiling on the client?

You can do this with Handlebars. I use a Grunt task to pre-compile all templates and then I just bundle them with the much smaller (it adds about 1kb or so) "vm" version of Handlebars.

Locally, I use the full version of Handlebars and regular non-compiled templates.

Same here. I had some trouble with partials until I realized that you can just compile everything as a template and then do

    Handlebars.partials = Handlebars.templates

I use:

    Handlebars.registerPartial("Foo", Handlebars.templates.Foo);

Your approach is kinda amazing. Makes me wonder why not all templates are automatically available as partial by default. Are there cases where you wouldn't want that?

I think the reason is Handlebars is supposed to look and act like Mustache and templates aren't named in Mustache, so they don't get named in "regular" Handlebars:

    var getHtml = Handlebars.compile("<div>yad yada...</div>");

And then following the rule of least surprise, if templates aren't partials during normal usage, then they shouldn't be with precompilation.

At least that's my guess. I didn't figure out my trick until digging through the source to figure out what was different. (Not much - just how/where they're stored & referenced.)

Ahm... yea, I forgot that I put them into `Handlebars.templates` myself if they aren't there yet (i.e. during development).

Perhaps I should have been a bit more specific. I don't really care so much about how you do it whether it's with jade or handlebars + grunt or whatnot (I happen to like jade). The main point is that pre-compiling and sending JS functions instead of strings is way faster and makes more sense.

Does anyone going through all this this know how to manipulate the DOM directly? It's ridiculously simple.

The whole approach strikes me as odd, unless I'm missing something.

You are missing something.

My team uses Handlebars templates with Ember.js. We build templates with placeholders for bound values or other view templates. When rendered on the client side, a change to a data model causes an automatic change in any templates that are bound to that data. In jQuery there's a potential for forgetting to write code to update a value or part of the page. With this templating system, it's all bound and auto-updated.

Another reason that we use templates is that we put all the template files up on a CDN. The only thing our server serves up is an empty DOM (just a body and a div), and the templates are all built into the DOM on the client side.

Lastly, managing a bunch of small tmpl files with the HTML we want is a lot easier than trying to embed HTML into JavaScript strings to use with jQuery. We just write up some HTML and an Ember view that renders that template in the right place on the page. Organizationally it's very simple and easy to maintain.

Direct DOM manipulation does not mean that you render things server side and then manipulate them. It also does not mean that you embed HTML in JavaScript strings. The difference is whether you use the DOM APIs (with or without jQuery) to create and update elements, or template libraries and innerHTML.

This is impractical with any JS-heavy application; where you are working primarily with javascript data structures and not the DOM.

It also leads to jQuery spaghetti code. So many people use jquery like a shotgun, to solve every JS problem, when better - and more targeted - solutions can be tailored out of small modular libraries.

You only get jQuery spaghetti if jQuery is your method of choice for manipulating the DOM. Most of my work is extremely JS-heavy applications, and I've never found it necessary to use string-interpolating templates. I have found it very useful to use a hand-rolled "template" system based on DOM structures, with data-attributes indicating where the templating engine should insert stuff. Might turn it into a real library someday, but it's really simple, so I don't see a whole lot of point to doing so yet.

It's true that manipulating the DOM is easy. The template thing makes more sense and becomes somewhat of a necessity when you write apps with data binding. As with about any framework, you can definitely do the same thing by just hand-rolling everything. But you wind up writing a pretty significant amount of custom code for every page. Using a framework you trade off highly efficient (but one-off) code for something that is more re-usable.

Can you explain how templates make data binding easier? Working with the DOM means that you immediately can save a reference to an element, rather than generating the HTML and then traversing that HTML to get the element you want to change.

It's kinda a different technique altogether. Instead of writing code to respond to changes and then update specific DOM elements, you just create a template that has usually some kind of template variables/placeholders that the template engine will populate with data.

Then you "bind" that to some kind of object. Whenever the object changes, the framework then re-renders the template. So the effect is that when your model changes all of your UI components on the page just magically refreshes themselves.

Probably not the best explanation, but hopefully makes sense!

A simple helper function makes DOM manipulation non-painful as well. One example is https://gist.github.com/1532562 or my fork https://gist.github.com/3524145

It's used like this:

    el ( tagName, properties, children )

Working with data structures is easy, children is just an array which can be the results of something like data.map() turning the data into elements.

This is very effective. After a while this is basically as writing html.

An added benefit is that it's very easy to attach events or other javascript properties to any object in the hierarchy.

Don't know the current state, but for a while it was a lot faster to just replace the HTML source (innerHTML I think?) than to manipulate the DOM. Just the way the browsers were implemented. No idea if the situation has improved by now.

Here are some benchmarks: http://jsperf.com/dom-vs-innerhtml/18

TLDR: It depends, but DOM manipulation tends to be faster in newer browsers.

Either way, it looks to me like this approach has to manipulate HTML strings to find a {{tag}} like that. Which seems like it's a lot more work than this:

and then adding a child text node or such to that element. Whether you do that via InnerHTML or CreateTextNode doesn't matter, either seems faster and easier to me than doing a string substitution.

But I don't know, I never really got the appeal of the framework of the week. If it helps write decent web sites, then I guess that's good, but I've noticed that simple sites now are ridiculously slow and I'm guessing this is the reason.

It seems faster, but unfortunately it isn't (or wasn't). That's just the thing. When I started, I also thought using DOM was a no-brainer, but there are all those tiny speed optimizations people make...

Do you build websites using echo, Response.Write or whatever your language of choice is?

The reason people use templating languages client-side are exactly the same as they use them server-side, it's much easier to maintain a larger app.

Most template languages are extremely lightweight and blazingly fast, slow websites are 99% of the time due to sending too much content or slow server-side processing.

You could also ask the same question about server-side templating: Why do we string-interpolate rather than manipulate the DOM? DOM manipulation is more robust and far, far more secure (you can auto-escape variables as you know their actual context).

I can think of a couple of ways to do that but it hasn't taken off largely yet:

https://www.facebook.com/notes/facebook-engineering/xhp-a-ne...

http://liftweb.net/

There exist purely DOM-based template engines, such as Batman's one: http://batmanjs.com (The approach has its advantages but unfortunately speed is not one of them.)

This posting seems like basic knowledge, but I can see how people get lazy and don't do this step as part of their build process and just send things to the browser to do it for them.

Part of the issue is that the default bin/handlebars script doesn't really support walking a tree and outputting a directory structure with all of the precompiled templates, but I've got my own hacked version of it which does...

https://gist.github.com/3719225

  handlebars ./handlebars --min --outputDir ./js/tmpl

I also have a build script setup in Eclipse so that when I save the file, it automatically builds things... (similar to this)...

http://stackoverflow.com/questions/6645640/integrating-coffe...

I use requirejs with a paths configuration like this:

  handlebars: 'handlebars.runtime-1.0.0.beta.6'

This allows me to just write this in my CoffeeScript for each page on my site...

  require('handlebars')
  require('tmpl/org/requests')
  ...
  
  requests.html(Handlebars.templates.org_requests(requests: requests)

All of this works amazingly well and has really allowed me to segment my code and templates up into little sections for reusability. Also, no need for ever loading the compiler part of handlebars in the client even during development.

Judging by the response, it appears to not be basic knowledge. But, I'm glad to hear that you and others are taking a similar approach as I think it just makes a lot more sense as a concept.

Sorry, I meant the part about the requirement to precompile templates in order to have the fastest possible experience.

It's common sense once someone explains it to you, but when you're learning about all of this new client-side tech you tend not to worry about speed until later, and then it's useful to see posts like these telling you what you can do.

I guess this is why frameworks like Rails compile your assets for you - it's a "silly not to do it" task, but not everyone knows about it.

Right on the homepage of handlebarsjs.com...

"It is also possible to precompile your templates. This will result in a smaller required runtime library and significant savings from not having to compile the template in the browser. This can be especially important when working with mobile devices."

But nobody reads documentation... ;-)

Am I missing something on why you can't just include your "templates" as display:none snippets of HTML and then .clone() them from the DOM and change the fields as needed? Perhaps my needs just haven't been complex enough to need a JS templating engine.

Wondering the same here. And what about caching the template library?

I use a JS templating engine (Handlebars with compiled templates).

When you have 3- or 4-level nested templates with 50+ fields to fill in, it's much easier to write (and especially to maintain) the snippets of HTML with {{name}}-type template tags, rather than writing the code to query the DOM and replace elements as needed. Especially if you have a designer who's comfortable manipulating HTML elements but allergic to JavaScript.

50 fields in a single template??? I use the parents technique because it punishes you for making gloat decisions like that.

From an accessibility standpoint, this is a bad idea. Not that long ago, you had to make considerations for clients without CSS support (e.g., text-based browsers or screen readers). Across the board, things have gotten significantly better but there is still the issue of a stylesheet not loading (and mixing markup and CSS isn't great so that shouldn't be considered a reasonable alternative).

If a renderer comes across a script tag it doesn't know how to parse (e.g., a script of type `text/template`), it doesn't do anything with it, however it remains the responsibility of the markup renderer and, therefore, you're not relying on something else (CSS or JavaScript) to hide it.

Not really, if you add style="display:none" inline, or use the `hidden` attribute, it will be correctly ignored by all screen readers.

Right but, as mentioned, inline styling is mixing markup and styles so it's best avoided (plus you're still relying on CSS to hide the content).

Regarding the `hidden` attribute, there is no support for it in IE (at least up to and including 9).

I know a lot of people argue against mixing markup and styling. As one of those, and since you seem well informed, how do you feel about the various css frameworks like bootstrap where the class names are essentially style descriptions?

There is a proposed standard called Web Components[1] that uses an inline-HTML approach (introducing the <template> element). Similar templating is used by Google's Angular.js, Knockout.js and other frameworks that implement observables/view-model binding.

[1] http://www.w3.org/TR/2012/WD-shadow-dom-20120522/

This can work, it's just a slightly more complex programming model, as it couples the template population code with the DOM structure. Whereas a template is effectively a cleanly-defined API in which you just pass it a set of key-value pairs.

DOM manipulation has traditionally been slower than innerHTML, which is another reason people might have shied away from this approach. I believe the performance gap is no longer clearcut though, so the argument may not hold for recent browsers at least.

Hulk Hogan generates JS functions from your templates too: https://github.com/twitter/hogan.js/blob/master/bin/hulk

You should probably never be parsing templates on the client.

We use RhinoJS to compile our Mustache templates with Hogan, since we don't have a NodeJS environment to compile this in.

I put together this gist as a quick how-to: https://gist.github.com/3719753

I remember a time when template engines ran on the server and sent finished HTML pages to the clients. What happened to that and why have we adopted methods such as this? Is this to balance load away from the server?

We've moved the entire MVC + routing frameworks all into the client in one go. Now it seems we're experimenting with which bits of work can still be delegated to the server. Pretty interesting times really, it all still feels very experimental.

As someone just learning Backbone.js this stuff fascinates me :)

Or we're just on the next iteration of the endless thin client -> fat client -> thin client -> fat client -> ... trend change. Now with 100% more bloat.

1) Yes to balance the load and to save bandwidth (if you have the same engine on front and back end you can make that decision late in the process or change your mind and move stuff between the front and back end depending on how the performance comes out)

2) These days more work just happens on the front end. With some web apps the back end is a pure data store and all the GUI is rendered on the front end. Doing a round trip for each front-end change would make the UI to unresponsive to be useful.

I have no idea why people don't do this so much any more. JS templating is crazy. Everyone knows JS DOM manipulation is slow on most current and all legacy browsers. Whereas string manipulation on the server is stupidly easy and fast.

Well, as someone who has seemingly missed the client-side MVC revolution, I still use server-side templating. And it's great, especially over slow conections. And it degrades gracefully when JS support is unavailable.

> when JS support is unavailable

That, yes. I noticed that my CPU load was noticeably high browsing some sites. Nothin special, just news sites. I found that quite annoying so I activated NoScript, suddenly the load was gone. The problem is, that NoScript seems to "deactivate" half of the web for me these days, precisely because of client-side templating.

There've been some valid reasons stated here though. However, the site I'm working on uses server-side templating, that's why I was asking in the first place.

While it can lower server load (though I doubt significantly in most cases) and bandwidth use (in some cases, where more data is present than the template chooses to display, it can increase bandwidth needed - but that is usually a design/tradeoff issue elsewhere), the main reason for pushing more to the client is UI responsiveness.

A lot of stuff is being pushed to the client now to reduce the amount of crosstalk between client and server, so things feel snappier for the user, with the goal to be to talk to the server when absolutely required (i.e. to save data or application state, to request new data or check the data the client has is up-to-date, or when some of your business logic needs to make a decision the client-side can't be trusted with).

If you are making display decisions client-side then your template engine probably needs to be client-side.

Web development is 99.999% based on fads. This is the current fad.

Closure templates does this. It's never occurred to me that someone would do it any other way.

The Google Closure javascript framework precompiles templates, too. https://developers.google.com/closure/templates/

I've written a Jinja-to-JavaScript compiler:

https://bitbucket.org/djc/jasinja

It reuses the Jinja front-end, it really only replaces Jinja's code generator and supports a pretty large subset of Jinja. I think this is particularly great because it allows you to switch from server-side template rendering to client-side rendering of templates piecemeal, or use both with the same templating language.

That's kinda neat. I assume it only supports built-in filters and control structures?

The compiler output looks roughly like this:

  var Jasinja = {
    "filters": {
      "attr": function(obj, name) { return obj[name]; }
    },
    "tests": {
      "lower": function(val) { return val.toLowerCase() == val; }
    },
    "templates": {
      "test": {
        "macros": {},
        "blocks": {},
        "render": function(ctx, tmpl) {
          return "a";
        }
      }
    }
  };

So you can easily add some filters by setting Jasinja.filters['myfilter'] to a function.

So. Write your templates in your server-side templating language. That way you can serve full html for initial page loads and compiled-to-javascript versions to allow Ajax updates.

Something like this: https://github.com/comolongo/Yz-Javascript-Django-Template-C...

It would be nice to add a layer on top of this that did partial page updates, managed the json views and the history push-state too.

It might be possible to have code where you just transparently wrote normal server-side views and the framework intelligently decided whether to render on the server or client and handled all the plumbing for you.

Thinking about this some more and here's me clarifying things for myself...

There's basically 4 strategies:

1. Give the client a full html page

2. Give the client pre-rendered html snippets

3. Give the client javascript that can render html and replace values without having to do much computation

4. Make the client do the template rendering probably using a library such as mustache.

So...

1. Should be the default and the fallback for dumb devices such as spiders and old browsers

2. Is ideal if the page doesn't change much - typical content heavy sites. You have to have proper urls and history management though.

3. is optimal for "it's an app not a website"

4. is only OK if you know your clients have plenty of CPU to spare. So not mobile basically...

I use Grunt (http://gruntjs.com) to compile them for production.

        handlebars: {
            compile: {
                options: {
                    namespace: "JST"
                },
                files: {
                    "dist/debug/templates.js": ["templates/**/*.hbs"]
                }
            }
        },

Then you can concat templates.js with the rest of your JS and your template functions are ready to go!

https://npmjs.org/package/grunt-contrib-handlebars

For a multiple page website, it seems like it is a waste to serve up all of the templates for the entire site. I try to serve the template that is specific for the page.

I'd be really happy if someone would contribute template compiler adapters for Flour[1]. It's pretty straight-forward.

https://github.com/ricardobeat/cake-flour/

You can have HTML linebreaks in JavaScript and HTML literals!

https://github.com/laverdet/js-xml-literal :)

that's cool.

That's a pretty awesome hack.

Are these template pre-compiled by the server at runtime or is there a build process before deploying the app to the server?

If it was happening at run-time, I'd probably rather just have the client do the compiling instead of the server. Since the client isn't going to notice a few extra milliseconds, but on the server that can add up under high load.

SoundCloud also precompiles their Handlebars templates. In general, it seems like a good idea.

http://backstage.soundcloud.com/2012/06/building-the-next-so...

This is a great way to do client side templating. I use Sam Stevenson's stitch and eco templates to do this, so I can require a template whenever it's needed.

Is there similar solution in the .NET world? I am using Knockout.js with underscore templates, and I would like to give this technique a try.

It's a technique in a lot of worlds: http://en.wikipedia.org/wiki/Partial_evaluation

(BTW, I just mean this as informative. I wish more people were aware of this idea, and I wish more mainstream languages would make this easier.)

You can just pre-process them with node and then use the generated template functions in whatever app you want. It's just client-side JS at that point. This isn't unlike pre-process SASS, or SCSS, etc.

Any idea on how client-side templating affects search engine referencing? Do crawlers always execute the javascript?

I use Brunch.io to do this. It precompiles all my templates and adds them to my app.js file.

Anyone have thoughts on using coffeekup?

I am sorry, maybe I am missing something important, but what is a logical difference between what you propose and concepts like ASP, PHP, ASP.NET? I am using templates in all pages, but only because data is known only on a client, i.e. I get data from AJAX call and fill-in a template (jsrender, knockout, others)

People have this belief that server side templates are slow.

For certain applications, they are.

If I have a single page web app, I don't want the overhead of returning markup. I just want JSON. And when I get raw data back instead of a string of HTML, I can be a lot smarter about responding to user input (e.g. optimistic updates).

Well, you send something to the server, then the templating happens, and then it's sent back to you. Even indefinitely fast templating would be slow due to network latency.

Also, sending JSON back and forth is kinda nice because it's fairly compact, easy to handle on both sides, and additionally this particular API may be used for other purposes, too.

Is this said in good faith? There are lots of reasons for client side templates, but I rarely (never) see anyone citing it because rendering on the server is slower than rendering on the client.

Client-side rendering probably isn't faster than server-side rendering in terms of latency, but it is cheaper. You save on server processing and network bandwidth.

Rendering speed isn't the reason server-side templates feel "slow" though: if you do server-side rendering then every change of state on the client needs to round-trip to the server in order to fetch a new HTML representation. Client-side rendering lets you "cheat": just render the new state and synchronize with the server after the fact.

Right.

How would you make asp.net rendering work with Knockout.js?

This article is about rendering HTML using template engine on the server. I believe this can be (and probably should) be done using existing server side engines, while javascript templates are to be used on the client. Of course if someone is using node.js as a server, then my arguments are not valid, but choice of server is another huge subject to discuss.

He's missing the point of client-side templates.

care to elaborate?

I think what he means is that its pre-compiled on the server, but is forgetting that is it client side templating because you are inflating your templates with data and adding them into the dom in the client as opposed to doing it on the server. pre-compiled doesn't mean server side.

Reply on news.ycombinator.com