Node.js - Convincing the boss guide
nodeguide.com
nodeguide.com
As a web developer, you sit on the intersection of engineering, design and art. You're lucky enough to live at a time where there are numerous production-ready tools with which to do your job. If you choose a tool that you're excited to explore, with an active and fast-moving ecosystem, your excitement and enthusiasm will make you happier and more likely to do a better job.
Someone down the thread said this:
> So far I don't see any benefit in node.js over mature enterprise frameworks.
If I was only allowed to work in ASP.NET, I'd be looking to change careers pretty fast.
Don't underestimate the value of enthusiasm - especially if you're a boss.
Indeed, and your advice is wisely directed at bosses. However, it's a bit harder (though not impossible) for a developer to use this argument successfully: "Let me use Node because I'll be more excited about the project."
Especially if you don't live in the US.
What happens when you assign 2 developers on a project that they are very passionate and enthusiastic about because you allowed them to use, say, node.js.
Only to discover in a month or two that they spent 200 hours more that they would if they used something proven and tested. That's several k$ out of the window and maybe milestone not met.
Sometimes it's worth it, sometimes it's not.
And regarding Node: it rocks, just try it and you see what eye opener you missed before.
Some people tried it and still couldn't see what they missed. I wouldn't call them blind afterward either. If it fits your taste bud then great, if not... no biggie.
With the flip side of that being ....? Hurry up and build it in whatever's new and cool before their enthusiasm for it fades and everyone's on to the next next thing?
I too, as a hacker, like to experiment with new stuff, and find that it's motivating, but thinking about how to manage a project for the long term is also worth considering.
From my point of view, Node.js is shaping up to be the php of 2012. It's proponents claim it's easy and fun and it's in a language that they are comfortable with. Meanwhile it's detractors see no real advantage over existing or other upcoming languages. Don't get me wrong, I'm not claiming it's quite as bad as php as that's something very hard to achieve, but equally I feel there are much better solutions.
I'm not at all against new languages, but I'd sooner invest in a new language that brings something more serious to the table. Perhaps the language is more concise so that we can write less code. Perhaps it's really clean with high performance and a steller C FFI. Maybe it leverages exiting mature eco-systems of software such as the .Net or JVM stack. Do you not find Scala, Python, or Clojure fun? I would hope you do!
Node isn't a new language. It's javascript. Node just gives javascript the rest of the web stack, conventions, and a package manager that feels like Ruby gems. Javascript already is the best solution for all the problems we're currently forced to use javascript for. It's the language the browser understands.
I'm not here to be Node's white knight, but Node isn't just Yet Another Thing or Yet Another Language. It's javascript and, to me, it's making javascript into a language with an ecosystem similar to what I'm used to with Ruby. Javascript is already out there in the wild solving problems that Ruby/Python/Clojure can't.
Examples?
However, if you really do insist on having one language everywhere, you can also compile Clojure down to Javascript using a subset of Clojure called ClojureScript.Unlike Coffeescript, ClojureScript uses Clojure semantics and is not just a different syntax for Javascript.
This doesn't save you from all the odd performance edge cases of Javascript, but at least it gives you conciseness and semantics of Lisp in the one situation where you have no choice of language: the browser.
I can't muster any enthusiasm over Node, because it's really more-of-the-same-but-better. With an attitude like this it's perfectly reasonable to ask "why exactly it's better"? Unsubstantiated enthusiasm in IT is evil.
We use it at a digital agency and it's been fantastic. I'm a big fan of what both Python and ASP.NET are doing at the moment, and although I have tried Ruby and Node.js I really couldn't see myself wanting to use them for a real project.
In fact, I'd go so far as to say that enthusiasm for mere new-ness is one of the more pernicious, counter-productive parts of modern hacker culture. Rich Hickey implied this at his RailsConf keynote when he talked about how great we are at judging benefits, but terrible at judging trade-offs. Here's a man who's certainly not afraid of newness itself (I mean, he single-handedly released a new JVM hosted LISP) but I think he has his head in the right place about the cost of new, and what something must provide to justify that cost.
For example, it makes absolutely no sense to use Node and then layer on top of it a web framework that is an isomorphism from some other environment.
Last but not least, I think the key reason we have different perspectives is age. Hickey and I are about the same age. I assert that anyone who trumpets the inherent value of "new" hasn't been around long enough to understand that there is a never-ending stream of new things and that, as Teddy Sturgeon famously quipped, 90% of everything is crap.
The real market for technology adoption should fueled by demand for simplicity, rather than demand new-ness (something else Hickey strongly implied in his talk).
The point I made is that enthusiastic and excited developers are valuable. Developers tend to find learning exciting. They could be interested in learning Rails or Django for the first time, trying out Sinatra when a full Rails app seems overkill, or exploring Node for a small and non-critical part of your architecture. Hell, they might be excited about learning PHP.
Allowing people to explore their curiosity brings material benefits. It doesn't have to involve new technology, and I didn't suggest that it does.
As an aside:
> The real market for technology adoption should fueled by demand for simplicity, rather than demand new-ness
I think part of node's mass appeal is that it has introduced an element of simplicity to a generation of developers who have grown up around bloat.
Perhaps you carefully avoid it, but you also strongly implied it. Consider:
>You're lucky enough to live at a time where there are numerous production-ready tools with which to do your job. If you choose a tool that you're excited to explore, with an active and fast-moving ecosystem, your excitement and enthusiasm will make you happier and more likely to do a better job.
You're optimizing for programmer excitement. There's a famous graph of programmer excitement curves floating around on the net, with a peak at "new and shiny" stage.
>Developers tend to find learning exciting. They could be interested in learning Rails or Django for the first time, trying out Sinatra when a full Rails app seems overkill, or exploring Node for a small and non-critical part of your architecture. Hell, they might be excited about learning PHP.
I'm not arguing the truth of what you're saying - it's obviously, painfully true that developers absolutely love trying out the new hotness (sometimes even if it's just new to them). I'm simply asserting that this is bad.
I at no point said that this should be the overriding factor. I said that the need to consider developers' personal goals is often understated, and is one of many pieces of information worth weighing when making a decision.
"I think part of node's mass appeal is that it has introduced an element of simplicity to a generation of developers who have grown up around bloat."
Who are you talking about? Right now I'm dealing with a generation of programmers that came up on ruby. Node isn't simpler than sinatra. In fact, we just tried to use node for a "small and non-critical part of your architecture". You know what happened? It took much much longer to finish because it needed to talk to an API that uses digest authentication and node's http library doesn't support that so we had to roll our own.
That to me was the point of his "easy" vs. "simple" dichotomy. Using the tools you happen to be familiar with is doing what's easy for the developer. Hickey argued that you should instead accept a bit of a learning curve if it will simplify your program in the end.
Now, I've never used Node, so no idea if it simplifies or complects web programming. (My current projects are in Haskell/Snap; so you can guess how I feel about learning curves.)
Though, I note that at the time, those were two mature and and well pedigreed technologies.
I assert that anyone who trumpets the inherent value of "new" hasn't been around long enough to understand that there is a never-ending stream of new things and that, as Teddy Sturgeon famously quipped, 90% of everything is crap.
A big reason for that are "pop cultures," which are defined as cultures/sub-cultures whose rate of change far outraces their rate of actual progress. Unfortunately, programming as a whole is one of those things.
The community
It feels like Rails in its beginning—so many talented folks providing great solutions. Just check Express and related npms like Jade, Stylus, etc. They look like similar Ruby gems but if you look into the details you see people that go further and pushes the boundaries.
Some examples: just check how beautiful designed the Express error messages are—nothing crucial but it feels good working with people who are passionate. Or check how lean Stylus is compared to SASS, LESS, SCSS or how Jade goes beyond HAML with mixins etc.
It's hard for me to articulate this enthusiasm I have with node but as much as I appreciate the Rails world I don't feel passionate with Rails anymore (it feels like my Cobol class in university).
These guys nailed my point pretty good: http://www.perfectline.ee/blog/optimizing-ruby-on-rails-appl...
What is node equivalent of: "rails g resource User email:string password:string"? AFAIK there is none. At least not one that would take care of migrations, validation, security, logic and views.
And if I want to use postgres? Sure, there's a npm package. But it's for queries, I still need to define models and validation manually.
....
http://stackoverflow.com/questions/9897017/railwayjs-vs-towe...
But the point is another: the npm world is much more modular than the Rails mononblock, there is much less magic involved. You setup your own stack which is simpler than setting up Rails, rvm, gems. Rails isn't bad—it's great but getting off track or looking behind the magic can get a tedious task ...
As far as passportjs goes, that's pretty much it.
On the other hand, devise handles A-Z when it comes to auth: it sets up models, routes, views, validation, different built in plugins like remember me, different encryption types, retry limits on failed logins,...
Googling for tutorials also reveals the true situation. There are like 2-3 good tutorials on passportjs. On the other hand there's hundreds of articles/posts on how to use devise, how to implement it, how to use it with omniauth...
I could make a comparison like this for almost every gem/module and I bet 95% of times, rails gems will win over npm modules in terms od popularity, completeness, community support and in-the-wild usage.
I'm not trying to bash node.js here. It's a wonderful tool and I use it. almost on a daily basis. But only where it makes sense to me.
Oh and about the 'magic'. You have two options:
1: start with bare language and proceed to magic as you learn
2: start with magic and dig into nitty gritty as you learn :)
Which of the two produces results faster? To me it's 2.
You use Rails because you want and understand the magic, not because it's magical. You might be initially allured by the terseness of Rails, but the learning curve of what Rails is doing is why you use robust frameworks in the first place -- because you want shortcuts and optimize away from trivial things like form parameter passing and route generation. Learning what `resources :topics` does is "tedious" in a much different way than the tedium of scoping out:
app.get '/topics'
app.get /topics/:id'
app.del ...
...
app.namespace '/topics', ->
app.get '/', ->
app.get '/:id', ->
app.resources 'topics'
...And then another part of the community will complain that it's "too magical" while I do the Jackie Chan 'wtf' gesture because there's a crystal clear API and they'd rather form a Cloud Chorus against alleged "magic" than humbly accept that they might not appreciate the same things that other people appreciate.
Yes Rails does all that for you automagically but you can use a lightweight ORM like FastLegS. A model is as simple as this:
UserDao = FastLegs.Base.extend({ tableName: "users", primaryKey: "id" });
To expose it:
app.get("/user/:id", function(req, res) { UserDao.find(req.params.id, function(err, result) { res.json(result.rows[0]); }); });
Let's not forget about unit tests. I can run well over 100 test cases in just over ONE second! Rails is PAINFULLY slow at this.
I've done enough Rails to know how productive it is, and there are many things I miss. I'll argue though I'm more productive with node.js since we do most GUI with Backbone on the client. I can share code between client and server, for example validations.
http://rakeroutes.com/blog/the-problem-with-completely-clien...
Other than that, surprisingly sensible advice.
I'm actually using node for a side project right now as part of re-learning web development (been back-end the last few years), and using javascript as a client language, server language, and db query language via mongo is a neat environment. Also, npm rocks. But don't listen to anyone who tells you they use node+mongo to handle their WEB SCALE PERFORMANCE. If they really had that problem, they'd be better off using the java ecosystem or, if they really wanted to go hardcore, using a lot of C. But Java's unfashionable and C's ridiculous to write a web app.
AFAIK, Node's cluster API, which can spawn workers for each of a machine's cores, will, when delegating requests to a worker, factor in OS-level core utilization info.
Again, I'm not saying node sucks, I'm using it and I like it. I'm just saying if you really need to crank out performance, use java.util.concurrent, or maybe Go. They both offer event-driven i/o if you really want it, and have better threading models.
Anyone with experience in both care to contrast them? Their problem domains seem to overlap quite a bit.
The other big downside to Node is hiring. You have two types of "JavaScript" developers. People good with JQuery, and people who have written JavaScript books. There is extremely little middle ground from which to hire good to awesome developers.
Go is the C replacement. Node.js is going to be a Ruby replacement one day.
I've recently rediscovered the joys of static typing so that alone inclines me much more toward go.
I wouldn't say that Node (i.e. V8) is a bad use case for CPU-heavy apps. In math/statistics for example, V8 stacks up well against other scripting alternatives. See e.g. the benchmarks here:
Of course it's still not as fast as C++, but not terribly off either. Expect dramatic improvements here thanks to the push for making web apps faster.
More to the point: Node does nonblocking i/o but cpu work is in fact blocking. This means that a CPU-heavy request queues up all requests behind it, waiting to get access to the CPU. You're better off using a threaded architecture for CPU-heavy work.
In Node, you would have a process responding to web requests, and launch a separate process -- via a convenient child_process with built-in pipe communication -- for doing the heavy CPU work.
The only case where that wouldn't make sense is if you need BOTH a ton of concurrency AND heavy CPU work. In that case, threads would make sense as they would be cheaper memory-wise. But my point remains: CPU-heavy applications don't necessarily rule out Node as a feasible alternative.
PS: V8 was 40x slower, not 400, in that one benchmark result you picked.
What would we have to do in node to get the same level of performance? We'd need a front end instance delegating requests to 8 back-end instances for each core? Writing all these bytes over internal pipes and blowing L1/L2 cache all the time as the data moves across cores? At a certain point, isn't it just way easier and more effective to have one thread spinning an accept loop and a threadpool handling the requests, all within one process bound to a port?
As I said upthread, I'm using node for a side project right now and I'm liking server-side js for a variety of reasons, but squeezing every cycle out of the CPU is not one of them.
Sure you could, depends on what you are doing.. In many cases it's fine that the request takes a second to complete, but blocking the entire server for 1 second is not.
Guess you define "heavy processing" a little differently, but in event driven you can't do any (blocking) processing
To him, any little one of script written by the team (except his own) must be re-done in Node. Then, the code being "his", he'll upload it to his own personal github account. Meanwhile, we have to reimplement the new Node written tool.
Erlang is actually suited for soft realtime systems as well. I've never seen it recommended for hard realtime systems.
This guide is OK, but it's insufficient. If you're going to do a technical evaluation of the node/nosql/js-client stack for risk assessment, you're going to need to get much more granular than what this guide offers. This is a fine guide for a project lead, but this is somewhat useless for the system designer.
For example, in reference to "nosql+node+buzzword+bull$#!t", the advice is to use nosql when you "really" know it, otherwise stick with mysql/postgres. While the devil-you-know-vs-devil-you-dont is a fine project mitigation strategy, it's not a very good technical strategy. Maybe the guide's author assumes one would do so, but the nosql/rdbms comparison exercise is not trivial and requires you to do your homework. Understanding and contrasting the finer points of comparing nosql operations vs. relational data stores is much more useful than a blanket comment.
Like I said, it's a fine guide, but it's much more useful for those who aren't necessarily responsible for determining the stack's applicability to a given project.
It doesn't seem as fleshed out as the Ruby Toolbox (https://www.ruby-toolbox.com) and could use a bit more editorial for your purposes, but it's a good place to start.
Of course, it'll never go anywhere until whomever is maintaining it either allows for crowdsourcing of the data or tries harder, because the first couple of packages I looked at had very poor descriptions.
Coffeescript, for example, is defined as "Unfancy JavaScript". CS is one that I happen to know about, and by that definition, I'd have never ever switched to it.
Some of my favorites:
- async -- flow control
- stitch -- packaging client side scripts
- express -- web framework
- redis -- great Redis client
- stylus -- css preprocessor
- request -- HTTP client