Scaling A JavaScript Codebase | Hacker News Reader

Scaling A JavaScript Codebase

yeungda.com

37 pointsyeungda12 comments

This looks a little overengineered to me. I'm always suspicious when I see a lot of 'envelope' code but no examples of code that actually does anything -- it reminds me of the guy who wrote a blog post about developing a sudoku solver with TDD who got to the point where had a few tests, but he never finished the app.

In this case we're seeing a few snippets of code, but no real sense of the structure of the app other than to see the particular way that 'main' gets called. In particular we get no information about how they've solved the difficult problems, such as how to organize complex asynchronous activities. (That's the scaling problem in Javascript, Flex, Silverlight, GWT, etc that used to keep me up at night)

I'm also a little disturbed that he wanted to write a DOM abstraction layer. Today there are many libraries such as dojo and jQuery that abstract away some of the worst problems with the DOM and with XHRs. Most of us could complain about the details of said libraries, but I'm very happy that other developers are doing the hard work to ensure browser compatibility, which is a real, not a hypothetical problem.

Re overengineered, I'm on board with what he's saying for certain circumstances. I have quite a large HTML/JS app I've written, and it's currently a little underengineered, and I'd like to refactor to add a bit of what he's done. (I've already done a lot of what he's done.)

Re: async, check out jQuery 1.5's "Deferred Objects". It's quite a nice little pattern.

Thank you for your feedback. This is my first article and it's a bit confronting to see only negative feedback in a comment.

Sadly there weren't more examples and that I didn't address every piece of the puzzle here! I do hope that despite this, you were able to take away something useful from reading the article.

On overengineered, why not have a look at the value of each part of the structure described and remove what is not providing value? We found that each part was valuable to us.

On asynchronous work - I shied away from it since it's such a big topic. If it helps, our most complicated interaction was three concurrent asynchronous calls. We wrote a small state machine that was notified of the result of each call and managed the complexity of deciding what to do.

On the DOM, it would seem that the DOM bit left too much room for interpretation. Unless we had to trade of simplicity for performance, we always used jQuery to access the DOM and do XHR. We certainly appreciate and even take for granted the safety that such a library provides. The point was more about reducing the surface area of the application code that interacts with the DOM, for reducing complexity and having a well known part that is good at working with the DOM. And anyway, we never did it - which says something too.

I thought this was a pretty good read. Here at blip.tv, we have a few sizeable JS projects in the wild (our HTML5 player, our distributor dashboard, and our destination site), each with shared dependencies.

Keeping discrete files in JS is one step from bringing your work from quick hacks to complete projects. I disagree with a few of the patterns in the post, but I thought it's on the right track.

As far as handling many asynchronous events, we follow a naming paradigm similar to Apple's Obj-C. It works surprisingly well for JS. We deviate a little bit with global (page) events via our own fireEvent() methods. There are also local (component) events that communicate within pieces of a page itself. We maintain a loose coupling of all elements on the page and thereby increase usability. We still provide a messaging system for other components to listen to, which allows us to have strong and consistent integrations across pages.

I'll write a followup on exactly how we do our concatenation and minification in a later post. I think it could be useful!

could you give us some idea on how big your sizable js apps are? I suspect this is a case where quantity does turn into quality. I.e. as continues to get bigger problems morph.

Coffeescript makes many of theses things seamless. Reminds me of another article called Scala == Effective Java?

http://grahamhackingscala.blogspot.com/2011/01/does-scala-eq...

CoffeeScript looks great, but doesn't it mess up the workflow to have to compile in order to fix a bug? Or do people typically download CoffeeScript into the browser during development? Or is using it enough of a benefit to outweigh having to do the extra step?

I compile CoffeeScript on the fly while in development mode. In production, I cache JavaScript. Debugging JavaScript and writing CoffeeScript doesn't bother me much although it can get a little awkward at times. I simply consider CoffeeScript a nicer, less painful way to write JavaScript and nothing more.

I use a watchr ruby script to keep the js constantly compiled. It is publically avaible on github (the project is https://github.com/danielribeiro/EaselBugTracker). Look into the Rakefile, coffeecompiler.rb and compileall.rb

Using on the browser is a possibility, but it is experimental (and experimental on a experimental language can get really crazy). But coffeescript is rapidly moving from a experimental language to a more stabilished one.

And if you are using Continuous deployment, the compiler step can be used only explicitely on the deployment fase, while keeping the js compiler on the browser only. I however find usefull seeing the compiled js even when developing (the emacs mode for coffeescript also helps a lot here), as chrome and other browsers still only debugs js (there is some work in allowing it to debug compiled languages).

Running

    coffee -cw *.coffee

will watch all of the CoffeeScript files in a directory and compile them to JavaScript whenever they change, which is what I use for keeping things updated in development, and the production deployment scripts I run compile the CoffeeScript to JS before deploying.

Aside from having to set up a terminal running the compilation task when I start a dev session, the fact that it has to be compiled doesn't affect me at all.

The script I use watch the whole tree. And it compiles from a source tree to a dest tree. But this is a nice hack for those starting out.

Here's ./dev bash script for node.js projects

#!/bin/bash

coffee -wcl public/ && coffee -wcbl ./

Uses the local function scope wrapper for js files served to client side in the public folder and removes function scope wrapper for server side files in the entire directory tree.

Type ./dev and I'm up and running.

Reply on news.ycombinator.com