Aspects of Building a Node.js Application | Hacker News Reader

Aspects of Building a Node.js Application

engineering.gosquared.com

65 pointsgnw21 comments

First of all, nodejs is a great choice. Having to worry about just one programming language helps a lot. I'll add a few things I found out along my ongoing experiment.

1. Tests are essential in any project using a dynamic language; but don't write regular unit tests. Instead write a setup script. A script that sets up basic data in the store by calling various APIs. For example, setup your test users POSTing to /users. Whenever you need to go back to a clean database (which is, often) you can run this script. And of course, every time you want to test all the APIs.

2. Automate everything in scripts. Script your compile, test and deploy steps. Script emailing. Script your database backups, tar your image folders.

3. Refactor every time you get a chance. It is like having a bath. You feel like working on your app, code feels fresh.

4. Optionally, try coffeescript. The language often promotes good, declarative programming. Once you get used to the shorthand, you won't go back. Don't discard it after spending a couple of days on it; it might take a little longer depending on your experience level. Another minor benefit, since there is a compile step some typos are picked up by the compiler.

5. One of the big benefits of Node is that you could move things like rendering from server to the client (or the other way round) relatively easily. But still it is cleaner to just have REST services on the server and handle rendering on the client (with say, Backbone).

Great pointers, setup scripts and automating project chores are essential for creating a good workflow, saving time, and saving colleagues some hair when they get started with the app.

I've tried coffeescript but it didn't catch on with me. Perhaps I didn't give it enough time, but I got the feeling that it could get quite semantically ambiguous at times whereas if you stick to JS' native C-style form, the structure of the code is imperative and easy to follow. I also felt uneasy about what code was actually executing at the other end of the transpiler.

a) You can always see the source output and it is easily readable. b) CS really doesn't do anything magic in the output.

Give it more time, the gains for CS are worth any negatives you can conjure up. do ->, classes, simple for loops, equality, etc.

I agree with most of this - great tips. However, for almost all applications, you'll want to render on the server, and bootstrap data on the client so that the client can take over and begin rendering using something like Backbone. This can be done by dropping a big json object on the page for the client to read as it initializes it's Backbone (or any framework) application.

This will solve the issues you'll run into later such as

* SEO - search engines won't read empty pages (you can get around this with some hacks) * Performance - loading a full page is always much, much faster than loading half a page, downloading javascript, parsing javascript, sending off multiple ajax requests for data, and then finally rendering the first page * Accessibility - a version of your site that works without javascript is not only easy to build, but is easier for machine reading * Development cleanliness - building your site in layers (html / http only, layer on css, layer on more advanced css, layer on javascript) keeps your code from becoming too tightly coupled and is much easier to work with. Plus, this makes it much easier to work with less advanced browsers

It's actually quite simple to do, especially with node, where you can share functions to format data, or even run the same framework on both side if you decouple your code (Backbone's latest change to help abstract Ajax seems like a step in this direction.)

In working with my app, I've found that Domains http://nodejs.org/api/domain.html are crucial for wrapping 3rd party async code that can crash on errors.

Absolutely, although Domains are a new feature in 0.8 and will need some time for maturity.

When your nodejs instances are crashing several times an hour due to facebook randomly returning html instead of json and the 3rd party library crashing as a result, who has time for maturity? =)

Wouldn't a simple uncaughtException handler be sufficient to prevent most crashes?

    process.on('uncaughtException', function (err) {
      // log it
    });

From the documentation:

"Don't use it, use domains instead. If you do use it, restart your application after every unhandled exception!"

http://nodejs.org/api/process.html#process_event_uncaughtexc...

I've had a similar need in my app, which has been in use since 0.6. I ended up basically abusing child_process.fork() -- every call to a certain rather long-running, possibly exploding async function is made not from the main node.js process, but is actually spawned as a separate process from node. So if it explodes, the core app instance isn't affected. Of course I do some intelligent things such as kill these "worker forks" if they take longer than 30 seconds, and I check for various UNIX signals to watch if they die and react appropriately.

I also tie the worker PID to each user's session, so they can't spawn an infinite number of workers and cause serious strain on the server -- the server just kills the previous worker and spawns a new one if it's asked to run the function again.

I was actually thinking about opensourcing this, but I guess it's pointless now that domains basically do what I hacked together.

It's probably worth pointing out that there is now an increasing trend toward higher level frameworks like Meteor and DerbyJS which help blur the lines between the server and client and simplify the creation of real-time apps. I think in a couple more years they will start to replace Express as the de facto framework to use for medium to large sized projects.

Can't you just use Fabric or Capistrano for deployment?

I can't believe I used to SSH into my server to restart an app when I can now just run `cap unicorn_APPNAME:restart`

As said in the post, Capistrano is a good option but we found we much preferred deliver, it was easier to setup and meant we didn't need ruby dependencies on node.js apps.

We also used to SSH into our servers and do everything manually. Crazy times.

Genuine question (since I'm doing the same with AWS right now) -- I wrote a script which makes an API call to Amazon to find the name of each of my instances, then SSH's into them automatically to do the equivalent of a git clone or update.

Seems to work so far -- any reason why I might want to be using a more advanced tool instead?

Over time, you will add features to that script and it'll grow and eventually become 'a more advanced tool'. Everything starts from somewhere.

If it works for you for now then no real need to change it. A proper deployment tool usually makes it easier to deploy apps of different types and languages using a consistent set of tools.

What I do when working with SSH'ing to AWS instances is to give them a readable name (like nginx-1, nginx-2) tag, and then place those names as hostnames in my /etc/hosts, so I can then just run ssh nginx-1. I use a script around https://github.com/bjeanes/ghost to automate this.

The script looks nice, though if you don't want to have to use sudo to update /etc/hosts you can also just add entries in ~/.ssh/config. Downsides are that it only works for ssh-based activities.

We use a slightly modified version of deploy from visionmedia (https://github.com/visionmedia/deploy). Shell script, and works very well for us.

Of course, when doing releases that affect the database schema, we still have to login to the servers.

I generally use a node script (sometimes a CLI script like [1]) with the ssh2 module [2] for most deployment tasks.

[1] https://gist.github.com/4281277 [2] https://github.com/mscdex/ssh2

Why mention newrelic, doesn't even integrate into nodejs.

newrelic can be used for server monitoring, and they've got a nodejs probe in development http://try.newrelic.com/nodejs.html

Downside: expensive

Reply on news.ycombinator.com