Leaving JSPs in the dust: moving LinkedIn to dust.js client-side templates | Hacker News Reader

Leaving JSPs in the dust: moving LinkedIn to dust.js client-side templates

engineering.linkedin.com

134 pointseneveu41 comments

"All told, we evaluated 26 different templating technologies, scoring them on a variety of factors: DRY, JS library agnostic, mature, open source, easy to learn, documentation, flexibility, performance, and so on."

If anyone from Linkedin is reading - any chance of getting this evaluation posted somewhere? Would be useful even if it's just a "tech x factor" matrix without commentary.

Sure, we should be able to make it available on the blog site.

That would be great. I also would love to see the see the matrix.

Any change you are going to discuss in more detail how to use the js templates on client and server side?

Sure, we'd like to soon write more about this. Node.js and jvm based rhino are the two solutions we are trying out internally. We are building some of our seo use cases with dust templates and server-side rendering. It is quite easy and efficient to reuse the same template for both non seo and seo cases

I did something similar for a recent project. My backend basically only provides an HTTP API. All rendering is client-side.

It felt natural as a programmer to develop the backend as a "library" that was "called" by the client side.

Immmmmense productivity. All pages require auth and shouldn't be indexed, so I don't worry about excluding search engines etc.

Not sure why you're being downvoted, but I like this too. Your application should just be an API and pages should be tiny programs that call into this API. That way you can easily test your application and easily test your pages. And when you want to release "an API", well, you already designed, implemented, and tested it.

Here here. Long live the one page web app.

hear hear

heer heer

> Once written, the same dust.js template can be rendered not only in the browser, but also on the server using node.js or Rhino.

I'm relieved to see their staff know what they're doing, and even curl http://www.linkedin.com/in/example still yields content. An alarming fraction of devs and toolsets out there would have screwed this up so badly that their resources would be cut off from reuse by the rest of the web, trapped behind an unstable single-site API.

Can you please elaborate on this?

I imagine the parent means that building single-page apps that require a full DOM+JS implementation is bad for the Web, because it cuts content off from being accessed by anything other than a full modern browser running several million lines of C++.

See also my answer to this question about single-page apps on Quora: http://www.quora.com/What-exactly-is-a-single-page-applicati...

IIRC, you're Node.js evangelist? What's your take on Node.js (server) + Backbone.js (or dust.js as mentioned here)? Not for all, but for Trello like client apps?

An enormous pig of a browser would be bad enough, but you also have to run the client js served by site X, because nobody else has code that's always going to be compatible with today's revision of the site X API. And if you want any behavior that isn't baked into the site X client js, you're just boned. That's why I liken this trend to the client/server days, when you were stuck with one terrible client you couldn't fix—which is what I thought the web had saved us from.

http://www.linkedin.com/in/example doesn't use dust.js if it did you would see it imported when viewing source. Am I missing something?

It should be noted that dustjs is not yet updated for node v0.6.x. I submitted a rather trivial pull request toward the end of October fixing the problem and another user has submitted one since then, but the author has not responded to either.

This makes me question how much the project will be supported in the future - but I suppose having a large company like LinkedIn invested in the technology is a good sign.

See: https://github.com/akdubya/dustjs/pull/36

JSP is an archaic template language. I'm on a project with JSP on the front end; compared to Django, Jinja, or any of the other Python template languages it's really limited and verbose.

It's interesting that LinkedIn chose dust.js as it doesn't get much attention compared to some of the other JS template languages (e.g. Mustache & Handlebars.js).

Do keep in mind that LinkedIn was founded in 2003, before many of the web frameworks were released. LinkedIn also uses a custom JSP compiler, as at the time the official implementation lacked many feature.

That wasn't meant as a criticism of LinkedIn's adoption of it, more my own circumstances and the fact that frameworks are still using JSP by default in 2011.

Definitely wouldn't use it. Even for server-side Java, there are better templating engine. I am not a web developer, so I am not fully qualified to speak, but in cases where I've needed templating, I've found Apache Velocity and Freemarket to be good options for Java (but most of the time, I am templating, e.g., code to be auto generated or configuration files, rather than web pages).

I wonder what's the best alternative for server-side template generation on the JVM.

I used FreeMarker on a previous project, it's pretty cool ( http://freemarker.sourceforge.net/ ). There is also mustache.java ( https://github.com/spullara/mustache.java ). I heard about Velocity and StringTemplate, but never tried them out... I guess Groovy Server Pages could also be considered if one's company is open to Grails...

Anything else I should be aware of?

If you need pure Speed + compile-time checked Typesafe, then Japid is the nr.1. Very popular under Play! Framework community. The performance is superior, 2x - 3x better than FreeMarker.

Are you using some old version of JSP? Not unlike many other mature technologies, many times people are just victims of some arbitrary corporate standard and not the tech itself.

As far as limited, I looked at Django strictly for it's vaunted templating prowess and didn't see anything that isn't easily reproducible in JSP. There could be some magic there but I missed it.

For verbosity, sure, fn:toLowerCase(string) is technically more verbose than string|lower, but marginally so.

Version 2.0 (current version is 2.1). Apparently upgrading is not trivial (due to Spring framework or whatever... I don't know, I'm not familiar with the Java stack).

You mention an example where JSP fares best; there are a lot of examples that don't come out looking too great though, such as time/date/decimal conversion, the obnoxious syntax for if/else, etc. A lot of Django's other filters just don't exist in JSP.

But that aside, there are larger scale issues -- composition of templates via blocks in Django/Jinja, which is a lot easier and more flexible than Apache tiles; the lack of auto-escaping in JSP; and the forced XML syntax, even at the expense of valid HTML (see what it does to script includes or empty divs, without a  in between).

Doesnt this play havoc with caching - I mean if you had a single UI webapp (that in turn queries all other webservices to render a page), then you could simply heavily cache your UI webapp. But when your client side JS is directly querying individual web services, in that case you would have to cache each individual web service (not sure if this introduces inconsistencies vis-a-vis different web services).

Is this how Amazon - who reputedly has everything as services - renders its webpages as well ?

Not only caching, but right now I suspect this approach could have _serious_ SEO implications. It seems to run directly against the guidelines against cloaking here: http://support.google.com/webmasters/bin/answer.py?hl=en&...

(Not that I suppose LinkedIn are too worried about search engine discoverability of a lot of their "dynamic, login required" content, but I'll bet they're _not_ using this technique on all those highly optimised pages full of people's names - like the 4th Google hit for my name: http://au.linkedin.com/in/iaindchalmers )

It wouldn't have SEO implications because using javascript to load content isn't considered cloaking. (Cloaking is where you present something different to googlebot vs. a live user). It's well known that googlebot does run javascript now:

https://twitter.com/#!/mattcutts/status/131425949597179904

There's also some speculation that googlebot actually runs Chrome and can render anything that Chrome does:

http://ipullrank.com/googlebot-is-chrome/

Yep. I've seen that (and I've seen evidence googlebot executing javascript in my own logfiles).

BUT - If I have a client relying on SEO/organic ranking for their money right now, there's no way I'd be suggesting they move to a "html template with javascript loaded content" style site - not until I've seen a few other similar/competitor sites ranking well with those techniques.

In the blog post it mentioned that is uses node.js to render server side if the templates can't be rendered client side. I'm more interested in how they are able to determine that at load time.

I am surprised that many of the big sites are just now getting around to moving away from the server side frameworks for UI. The benefits of doing so are just so great that I am surprised that they ignored it for so long. Flattening the web stack makes development rapid again, UI engineers can stub out JSON files to emulate the REST back end and iterate through bugs so much faster. At many of the companies that I have implemented new style web front ends out TCO dropped significantly. I guess it's telling of the times when you are more shocked to find out that people are just now doing as opposed to the fact that they are doing it.

As well I enjoyed the article, I have not used dust.js yet so it was interesting to see how it works. Dojo has a good templating model and I tend to prefer it over some of the other offerings, it is good to see that others are getting into the space. Though I do prefer Dojo's widget based templating over standard tiles type templateing, I find that it makes the code more compartmentalized and reusable. More of a black box, where code and template can just be added to a page and it works.

Unless I am missing something, the bullet point near the end, "Progressive rendering", is not correct when it states that template rendering is asynchronous and parallel. Fetching of data, sure, but unless they are using web workers to execute the templates client side, and I do not see workers mentioned anywhere, this is not accurate. When a template is rendering, all other JS and UI will stop.

Having applied the same approach as LinkedIn on past projects and jobs (e.g. BN.com), there is definitely a lot to be gained if done right, and web workers, when available, ease some of the issues with rendering large templates and/or large datasets.

Curiously, we tried this and backed away to using AJAX calls returning pre-rendered html fragments in a recent project.

If you're building a mobile site, and want to target low-end Android devices (our "crappiest-phone-test-device" was a Huawei U8180), you really don't get a lot in the way of JSON parsing or DOM manipulation performance.

I wonder what (if anything) LinkedIn are doing for mobile?

http://engineering.linkedin.com/nodejs/blazing-fast-nodejs-1...

Nice link, thanks.

It doesn't address the problem we bumped into though. When a page needs a significant amount of content updated, sending that as structured JSON to a low powered Android device (or sometimes even an iPhone2 or a non S 3G) and then parsing that JSON and updatng the DOM was _way_ slower than sending back HTML and just replacing an elements innerHtml. For small tasks (like the personalisation {'name':'John'} example) we could make the JSON approach work, but updating a ~25row 4column list? Not on the cheap Android devices...

What do you mean by 'significant amount of content'? Inserting html is of course faster than rendering in the client, but the javascript involved here is not slow - I haven't seen anything over 15 ms yet for rendering a fairly complicated page (I'm using a resig-style microtemplate, which is probably faster).

Are you perhaps constructing the nodes on the document, instead of doing them off the tree and then inserting/replacing them all at once?

Does any know how is dust.js compared to Beebole PURE, an JS template engine too. They claim, PURE should be fastest JS template. ^^

As vybs mentioned we'll publish more detail on our evaluation process, for which performance was one of many features we looked at. It may be useful to start by asking what's most important for you/your organization (performance, productivity, ramp-up time, server-side rendering support, etc) since these priorities will likely be different from ours.

What about SEO and accessibility. If the content is sent via JSON it's relying on javascript, no?

Correct, the template can only be rendered on the "client" in cases where JavaScript is available. In other cases (including for the small percentage of users who have turned off JavaScript and mobile devices without JS support) the solution is to render the templates (turn them into HTML) on the server.

bvincent, could you please elaborate on this? How do you tell javascript enabled users apart from disabled ones? Is there any website discussing this you can point me to? thanks

JSP, I never knew. That certainly explains a lot, particularly how it meshes with my sense of anything-JSP being a couple of years behind, featurewise. I don't know why this is, if it's a language non-feature, or a benefit of open-source libs, Java coding culture, or what, but I've thought that LinkedIn was making some silly site decisions and being inexplicably behind the times for some time now.

Reply on news.ycombinator.com