Why we moved from NodeJS to Ruby on Rails | Hacker News Reader

Why we moved from NodeJS to Ruby on Rails

blog.targeterapp.com

239 pointsheroic159 comments

I really do think that, as time wears on, we will see more of these incidents of people switching from Node to another, more familiar development environment. It seems to me that a lot of people started using Node simply because of the amount of hype that surrounded it, and many of these decisions were not as well researched as they should've been.

Reading through this gentleman's blog post, it seems that, originally, Node was chosen for just one reason – he wanted to develop the entire application in JavaScript. Other than this, it does not seem that he took the time to research the Node development environment before deciding to develop his application in it.

With respect to the author of this post, the testing environment and the ease of developing a CRUD web application are things that should have been looked into before a single line of code intended for production had been written.

In addition to this, as several other commentators have already noted, Node is not directly comparable to Rails. Node is basically an event framework and standard library that bolts onto JavaScript. Node would be more comparable to something like Python's Twisted or Ruby's EventMachine. The author was probably using one of the frameworks available in Node such as Express.js, Railway.js, or Geddy.

Actually since the app was developed at startup weekend which gives you 54 hours to make a presentable demo, I'd say testing was not the first thing on their mind.

Plus if they already knew rails. I use node mostly now and wonder why people would use anything else. However if i had alot of expertise in rails i would probably use it for most everything as well until i was comfortable enough with node.

And it's generally nowhere near 54 hours... more like 42. We nominally started at 6pm on Friday night, but teams weren't formed until 9-10pm. Pitches were demoed Sunday afternoon, and 48 hours from the start on Friday - Sunday at 6pm, it was all over. I've no clue why they SW fixates on the "54 hours" number. 42 would be far closer to the truth, but perhaps more intimidating for people?

It seems to me a lot of this starts with "I have a general idea, and I want to get it off the ground ASAP" and transitions to "I have a focused idea, and can now revise the project accordingly"

I see nothing wrong with that, as long as everyone involved recognizes the cost of rewriting/replacing large chunks later. If you're on a super tight schedule and don't want to write anything but JS, and you can afford to make revisions later (assuming the thing even catches on!) it strikes as pragmatic.

I think you nailed it. Different solution are valid at different times.

Also in many cases, a website rewrite is really nothing complicated.

The former: Golden. The latter: Eh, not so much.

Indeed, planning for 3 months to make sure you've done no wrong is much better than coding for a weekend, and determining then if you've made the right decision. Funding engineers for 3 months isn't all that expensive and everyone always makes the correct decision after 3 months of planning...

From my personal experience, Node is a really fun platform to quickly whip up a working web site/app/service over the weekend. But I keep getting this nagging feeling that it's not sustainable for a long term project with a large codebase. Other than socket.io, I really don't see Node as a general purpose web stack.

Actually we have around 20,000 lines of node in production spread over about 20 services. It's perfect for SOA.

I only played with Node.js and I didn't like it. The event loop is cool for long-lived requests. It scales and so on.

However for the project I worked on, I decided to go with Ruby on Rails + Java Servlets.

You see, only some parts of an API needs to scale. But what about simple web pages like the viewing/editing of a user profile? What about viewing some stats that are auto-generated? What about the freakishly boring admin interface that every web app must have? What about deployment automation? What about just searching for a library that already does what you want and actually finding a good one?

Therefore I've built a Ruby on Rails app. Everything except the API that needed to scale was built on top of Rails. Then the API with scalability concerns was built as a couple of Java servlets. And I deployed the whole thing on top of Jetty.

The wonderful thing about Jetty is that it has support for continuations, so requests don't have to block on processing in case you're doing something expensive. You can just push that processing in an Akka actor and release the request until a response is ready. And Jetty may not scale as well as something custom built on top of JBoss Netty or Apache Mina, however a single Jetty server does scale to ten thousand requests per sec easily. And in case that doesn't satisfy me at some point, Netty and Mina are there, waiting for me to tap their potential.

And then with the wonderful JRuby-Rack integration, I could configure Jetty to select between the pure Java implementation and Ruby on Rails for serving, based on the URL. So everything, like the Rails server, the Java servlets for the API and the queue processing (Akka) is running in a single process. Which really, is freaking awesome.

So why use Node.js, when I can get the Rails maturity and ease of use, while escaping to Java in case I have special needs, such as extreme scalability or flexibility?

I could write a similar post, having used Node on a couple of projects before nodding to reality and switching back to Rails for a larger project.

For a "typical" web project, Rails simply has a ton of mature libraries that don't exist in Node, or are very early days. Sure, there might be a basic data mapper, but does it have support for testing? Automatic counter caching? Workaround on StackOverflow for a hundred things the framework doesn't cover by default?

Rails versus NodeJS isn't apples-to-oranges. Rails is an opinionated framework for web development, Node is a low-level evented framework. This is important because right now, Node has no Rails equivalent. It has Express as the equivalent of Ruby's Sinatra, but for a higher-level web framework, there are several possibilities and none of them the outright leader. This is the situation Python found itself in circa 2006, before Django became the de facto standard for web apps, and (speaking as a mostly-outsider) its seems to me the Python community is better for it. I truly hope a high-level framework establishes itself as the clear standard for Node development, presumably one which embraces the benefits of dual-side JavaScript.

There is a project coming up - railwayjs. I hope it changes the way people use nodejs!

I hope so, but with the caveat that it's easy to start a project like this and a lot harder to see it through. Various Node projects have shown promise, but haven't gone the distance. If one really gains traction, that's great for Node overall.

That's part of the problem in a way. Rails-style large applications are kinda poor by-design. Even Express suffers from this more than I'd like (express-specific plugins etc). Ideally Express "plugins" or extensions would just be thin wrappers around things that the rest of the Node community can utilize. I'm guilty of this too of course but that's largely due to time constraints.

Anyway, writing Rails for Node won't be anything revolutionary, it would just be... Rails on a different platform. Frankly lots of developers including myself prefer these smaller frameworks, sure some things might be a bit slower to get started with but ultimately you're not fighting a framework, it's just another block of lego.

I used Drupal for about 3 or 4 years, and you can get things going much faster than Rails even, but you'll spend a lot of time fighting the thing.

The problem is that Rails has been around since what 2005. Node has been around for a much shorter period and has only stabilized over the past year. So they just arent comparible in that sense. Plus node works differently then Rails and i admit it takes time to get used to that.

If you do take the time to learn node though all i can say is its just AWESOME! So lets see how things are when node is 5 years old or so.

Definitely, Node still has a lot of promise. It's not a competition. Well, it is to some, but I don't care about that. I just care about what do I use to ship something useful today.

When it comes to prototyping, anything goes - you should be optimizing for the speed with which you can deliver and market-test a product, and you should choose the tools that allow you to do so. The trick is having the discipline to pay back the technical debt that accumulates as a result of the fast, informal prototyping work.

I think the OP did exactly that, realizing that Rails is the better long-term solution for their engineering activity. I wouldn't say they "moved" from NodeJS to RoR, because that implies that that they chose a new technology to solve the same problem - they didn't... they chose NodeJS for prototyping and RoR for production.

I had the same experience transitioning from a prototype RoR app to a production app also written in RoR - I still did significant rewriting to improve testability/test coverage and make it more RESTful, but I didn't need to switch frameworks to do that.

> NodeJS is suitable for apps that do plenty of short lived requests.

I'm confused. Isn't Node's event-loop style programming ideal for long-lived requests? I.e. ones that, under a synchronous i/o, block other requests?

The OP probably meant short-lived with reference to CPU bound requests.

OP explained things improperly, then. It's an important detail, because Rails currently defaults to single-threaded. This makes an operation such as fetching and returning a twitter feed in a single request (assuming it takes hundreds of milliseconds to get a response from twitter's servers) expensive.

Only rookies do that inside the web server process.

The job of fetching a Twitter feed can be offloaded to a background jobs queue. With a little help from Nginx, you can free the Ruby process to take care of other requests until the response of that Twitter feed is ready.

Or you could simply deploy your Rails app on top of a Java server, by means of JRuby and forward that request to a servlet that uses the continuations support in EE 6, offloading the request to an Akka actor and freeing the pipeline until it is ready. Works great and you can even write everything in Ruby ;-)

Also, Rails does work with multi-threading.

Or you could just use node.

If all your app ever does is to fetch Twitter feeds, sure.

   Rails does work with multi-threading.

Ruby has no support for concurrency, no matter how many threads your interpreter is using. 1.8 had no OS threads at all, and 1.9 has a global interpreter lock. This is not solvable in the application layer (for example, by a framework like Rails): this is a problem inherent to the runtime.

MRI doesn't, but JRuby supports full parallel, concurrent threading. Which is brilliant.

JRuby is fully parallel with no global lock and many people choose it for deployment. Rubinius 2.0, which is in development, will also lack the GIL.

You're also very confused about how threading on Ruby MRI 1.9 works. First of all, pure Ruby code in 1.9 can and does execute on multiple OS threads, in parallel.

Also, the problem that the Ruby VM still has is that while executing native code, it does not allow a context switch unless that code explicitly informs the VM that it can do a context switch. This is in effect how the global interpreter lock works. The gotcha here is that native extensions that are well behaved, can inform the VM that a context switch is possible. For instance, the older "mysql" gem was NOT well behaved, blocking context switches between threads, but the newer mysql2 gem does behave well and works correctly in multi-threading.

Right now, if you start a new Rails 3 app, it will work in a multi-threading environment correctly and modern Rails servers are taking advantage of that, unless you install some older gems that haven't been fixed. The biggest problem is that you can't know what libraries are well behaved, but if that's too much of a burden, JRuby is a fully supported platform for Rails and doesn't share the same issue.

So like, seriously dude, do some reading.

Pure Ruby code in MRI 1.9 can run on multiple OS threads, but it can't run on multiple OS threads in parallel. Here's some reading you can do if you're unsure:

http://merbist.com/2011/10/03/about-concurrency-and-the-gil/ http://www.igvita.com/2008/11/13/concurrency-is-a-myth-in-ru... http://jabberwocky.eu/2010/10/02/where-is-the-global-interpr... http://en.wikipedia.org/wiki/Global_Interpreter_Lock

Yes: Ruby can allow native code to execute in parallel with Ruby code (although it doesn't always do so, as you note). But if you're under the impression that multiple Ruby threads can execute in parallel, you're wrong. That may or may not be a problem, depending on what you need Ruby to do.

JRuby can be a good option for parallelism, but it's also slower than Ruby MRI 1.9 and has an ecosystem that most Ruby developers will be unfamiliar with. Regardless, my point was that Rails doesn't magically "work with multithreading," at least for standard Ruby deploys.

"short lived" probably means that the business logic is lean.

This post would benefit from concrete examples rather than vague complaints. For example, if something is "very immature", why not include specific examples of bugs or issues that required upgrading to "the latest version"? Likewise, if Node's testing frameworks are "good" but "no match for … Django", explain what's missing or how other testing frameworks are better. Being precise with your criticism helps the reader put them in context, and also helps maintainers improve their projects. Unsubstantiated criticism merely generates fear, uncertainty and doubt.

Wow, Node is being turned away from because it's too new, and Rails is the destination because it's established?

I remember a few years ago Rails using the argument of re-inventing the wheel was the only way because there was presumably no other capable web framework out there.

We all know better now and I hope celebrate choice that's relatively equally capable :)

Maybe some day people will finally realize what's so awesome about Ruby on Rails. It's not the libraries and the fact it's been around for almost 8 years now. No, it's simply the language. Ruby is a fantastic language and an addictive one at that. Once you get a feeling of how productive and expressive it is, it's enough of an incentive to keep at it. The same goes for Python by the way, another GREAT language. One day people will realize that Javascript just doesn't cut it.

I think you totally missed my point.

I was just speaking to how Railists too often, do not openly consider or think anything productive, well structured, or maybe even great could have existed before, or at the same time as Rails for the web.

We forget the difference between a language and a framework as well. Too many folks short sightedly compare a language to a framework. Frameworks will always evolve and change, and few rarely do anything new or novel, just like languages may, but they're entirely different things.

And people switched away from Rails back then too :-)

http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_swit...

Is this article from 5 years still relevant?

I know you said 'back then' but curious to know whether one should stick with php.

Yes/no/it depends - pick an answer :)

I believe Derek kinda went back on this answer a bit and ended up taking up Rails again later on, but the issues regarding PHP being more practical and flexible for him at the time were pretty important. These issues will still be relevant for many developers and even as a Rubyist myself, I'd say Rails is certainly not the right choice for everyone.

The data is being written to the db, or being crunched, while the request is ending, and the browser can continue with the important stuff.

It sounds like a good fit for NodeJS for sure, but for what it's worth, even in a synchronous web framework nothing forces you to keep the connection open until the request has been processed.

I don't know off-hand how to do it in Rails, but say in Java Servlet-based frameworks you'd just call HttpServletResponse.close().

The point is that the rack middleware would be called for everything, which is very heavy for something so trivial

Of course, it's trivial to write request methods that bypass the rack middleware in Rails.

I didn't mean to imply that NodeJS isn't the best fit for the problem at hand. I was just trying to clarify that it doesn't take an asynchronous framework like Node to keep processing a request after the connection is closed.

Yeah, even with a simple Perl CGI, all you have to do is close STDOUT if you want the HTTP response to end, but continue processing the request.

I suspect -- and I may be wrong -- that the advantages of using Node differ between large teams and small teams. For my first startup, I had the budget to hire a single full-time developer. The was a substantial amount of work to do on both the frontend and backend -- with the backend development needing to get underway first, but the frontend work being much more experimental and needing to go through many more iterations as part of the development process (we understood this upfront).

My developer did his homework, and came to exactly the same conclusions about Node that this article highlights. So he began developing the backend in Python/flask. I'm sure that was a good choice for the backend, on general principle.

However, we began hitting problems when it came time to start developing and iterating the frontend in parallel with the backend: it proved to be almost impossible for a single person to handle the task-switching between two completely different languages, testing frameworks, etc. So backend development lagged a little, and frontend development lagged a lot. I'm fairly sure this set the project back by 3-4 months. It wasn't until we brought on a dedicated front-end programmer that development was able to pick up pace a bit.

So I can see how for larger companies, it is probably best to optimise the backend with something other than Node -- you will have enough people working for you that backend and frontend programmers can specialise. But for companies with two (or fewer) developers -- which know that they also have to be doing a quite substantial amount of javascript on the front-end -- then I suspect that Node's comparative small task-switching penalty is probably hugely valuable.

That's just my hunch, but it's lasted a long while. So as I get ready to hire a couple of developers for my second startup, I'm leaning towards using Node. It's a frontend-intensive application which is unlikely to ever have to deal with "web-scale" issues that more mature backends are needed to handle. And if it does ever approach "web-scale", then it'll certainly have the budget to hire additional programmers to re-implement the backend in something else -- or maybe, by then, Node will have evolved into something sufficiently robust. Either way, it seems better than causing ourselves a bunch of task-switching pain upfront.

I'd be really curious to see a critique/comparison of Node which addresses the relative merit (or lack thereof) of its low task-switching barrier -- particularly for small teams.

What it this "low task-switching barrier" ? I mean, even though, the language is the same, back end coding and front end coding are really different. For example, your front end development may have lagged because of lack of front end development knowledge, not because of this mythical task switching barrier that everyone is talking about !

Maybe not true for everyone, but I find when working on quick prototypes of web ideas, where I have to write both front and back-end code, the fact that they're both in the same language with node.js is actually a huge advantage -- much greater than I expected.

IMHO the main problem is that usually front end development implies XHTML/CSS integration and/or UI and/or UX, and that's a very different job from back end development.

I read this post as "your mileage may vary", gravitating toward "it doesn't work like we expected". One particular line caught my eye:

"NodeJS’ async everything wont have any affect on a CRUD app. Popular frameworks, provide very good caching and everything that is needed for your app to work fine even synchronously."

There is a lot to this statement that implies how an application is architected. Node.js is async at heart, and one can only build a successful application if you follow the programmatic model as it's designed. It doesn't matter how simple or complex an application may be, if a framework is being bent to work in ways in which it wasn't designed, the application may not work so well.

Reading between the lines, what I'm getting is a picture of a team that has encountered productivity issues with the current stack and is migrating to a more familiar stack to increase productivity. Nothing wrong with that tradeoff, but I always look to what future tradeoffs are being made: "cache everything and gets the work done ASAP" is great until you need finely-grained control of your queries.

I'm sure they thought well about their decision, but if asked to make a wager -- I'd guess that certain problems are being traded from one framework to another.

I think the point wasn't that "asynchronous is hard", it's that "asynchronous is just incidental complexity". CRUD apps don't need to be asynchronous, and node.js has a relatively impoverished ecosystem for such apps.

There are undoubtedly tradeoffs no matter what you use, but this seems like an eminently practical decision.

I'm not so sure about that (the async-is-hard part.) If one is making a decision of architecture change due in part to CRUD capacities within a framework, I'd guess the asynchronous programming model in Node.js is a little more than just incidental complexity.

CRUD operations within R/R might be just a few lines of code, but it's not that much more to accomplish the same with Node, Express/Railway, and any one of a bunch of template schemes.

The change certainly sounds appropriate for the team, but the basis of CRUD as a reason to change an entire architecture leads me to question how the team approached their solution with Node.

Unless you're taking the sunk cost of the node.js development into account, I don't understand your point.

If the core programming model for node.js doesn't help them in any way, and the ecosystem is less mature, then it kind of misses the point to say that everything is doable in node.js. Whether something can be accomplished is not a good metric for whether a platform is productive.

> If the core programming model for node.js doesn't help them in any way, and the ecosystem is less mature, then it kind of misses the point to say that everything is doable in node.js. Whether something can be accomplished is not a good metric for whether a platform is productive.

Hmmm, don't recall saying everything is doable in Node.js. I was speaking of CRUD operations.

I've had experience with many teams who have made similar architectural decisions based on broad, abstract data points, i.e. programming model, ecosystem maturity, etc. Frankly, making these types of decisions on those over-arching points rarely leads to good decision making. Most often, problems of one type are simply exchanged for problems of another, and it's usually caused by lack of real evaluation of system/architecture needs.

I'm not arguing this team shouldn't make this switch, just that the stated reasons don't pass the sniff test. I think the team prefers to programmatic model of R/R to Node, and feel they would be more successful going forward. But I would be hard-pressed to believe this transition is being made for most any reason beyond comfort and familiarity, as opposed to significant system limitation.

This is why it's good to have a lot of tools in your toolbox. Everything has its strengths and weaknesses.

The author seems to be confused about what Node.js is.

> NodeJS is a very young framework, and packages around it are very immature.

Node.js is not a framework. It is a JavaScript engine and standard library. Node is comparable to the MRI.

A standard library is nothing but the default framework built into the runtime system.

I don't see what you mean, the author is comparing Node.js to RoR so what you are referring to as a "framework", is not the same thing as the author. He is referring to a framework for writing web applications. Node is a "framework" for writing JavaScript applications that are compiled by V8 and executed on a *nix or Windows machine.

First thing is that NodeJS is not a JS engine, V8 is. Second, http://notinventedhe.re/on/2011-7-26. That is from http://nodejs.org/community/ which makes the terms quite right, i'd say.

Node is V8 and a standard lib. Node is not an abstraction around http (which is what RoR is).

"Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications."

From nodejs.org

Right, and "fast, scalable network applications" !== "http abstraction layer". Node doesn't even come with an http abstraction, it comes with the standard request/response objects that nearly every standard lib ships with.

Node is an abstraction around libev. Http just happens to be the one source of events.

Maybe I'm missing something but isn't node for chat/messaging, API's and other stuff that you might have used something like Twisted for?

Why would you use it over Rails/Django etc. for a content-heavy site (i.e. 95% of websites)

I use node for everything now myself for a number of reasons. The "problem" with node is that its still very young. And its a different programming model then people are used to. So its similiar to the whole RDBMS/document store argument. You use what works for your applicaiton.

The thing that annoys me personally is that they make it seems like there something wrong with node. And there isnt at all.

Does it provide any of the functionality that you get with a Django/Rails style framework? Or is it at a lower level?

Yes of course via express.js and railway and several other frameworks. Again node itself is just a set of libraries on top of of v8. So its not a proper comparison.

Personally i use express.js and roll my own framework essentially because i prefer to work that way.

There's are some web frameworks that are aspiring towards Django/Rails-style, but none that have a similar ecosystem of plugins and developers yet.

The most popular web framework, express.js, is a bit more lower level, and is similar to Werkzeug from Python or Sinatra from Ruby.

The important point in posts such as these is that execution matters, not the tool you use. If the tool you decide to develop with fits, then use it. If you find out later it doesn't work, then switch. Sure, the tool you use can affect your execution, but that only means you have to understand how you want to execute before even selecting a tool.

I think it would be a big mistake to read such a post and come to the conclusion that Rails is somehow superior to node.js.

My take on this is that it is more about getting the MVP out with "whatever" (in this case Node), than anything else.

I do this all the time. Some ideas get mocked up in python, others PHP, some in C#, etc. Then if it sticks, I look at the needs it has and pick the best tool for the job. Sometimes it Rails, others its .NET.

Though it is quite interesting to read about it from someone else's point of view.

This reminds me of Derek Sivers switching from Rails back to PHP 5 years ago :-)

http://www.oreillynet.com/ruby/blog/2007/09/7_reasons_i_swit...

Haha great find. Thanks. I agree, I think it would have been important to also say just how quickly Node.JS has developed.

I'm a firm believer it has a very bright future, but this is not a zero-sum game.

The argument he makes is that it boils down to maturity. One framework is still evolving and the other is more mature.

Exactly the point being raised i'd say!

I'd be curious which things were missing in Node. I realise gratuitous detail doesn't make for a great blog post but if Node.js is missing things that stopped development of an app it would be useful for the Node community to know what they are.

Mentioning "immature packages" and "testing" is fine, but I'd really like to know what makes testing on RoR better than, say, testing with Mocha.

I'd also be really interested in the experiences of trying to cache with Node. I think while it's possible to do it's definitely a place that could be improved.

IMO, things like a good ORM(or ODM) are still way too far off in NodeJS. I am aware of mogooseJS, but it is very far from the power provided by ActiveRecord and Mongoid.

Also mocha is powerful, but if you have a look at capybara and rspec, they are much more powerful and friendly.

The final thing in my opinion is the speed of development. With rails you go to devise, and you have authentication. I am aware of the presence of a similar authentication system for nodeJS but I am not very sure if that works with mongoose js and is as extensible as devise is. And then there is the rails console, which makes trying out things real easy and fun.

I started looking into libraries for a Node-based model layer this weekend. The two most interesting after my initial reading about are Tubbs [1] and Resourceful [2].

I won't have a chance to actually play with them properly until this weekend unfortunately. Anyone else have any experience with these two (or other options)?

[1] https://github.com/dandean/tubbs

[2] https://github.com/flatiron/resourceful

Personally id say Node and Rails simply shouldnt be compared in any way. They are just too different technologies. So node shouldnt even be concerned about trying to be rails at all. People are going to use whatever they want to regardless. So node should just do what its own users need. And thats it.

Anyone else find it ironic that this post is about moving from Node to Ruby in order to produce the admin interface to a Java hosting service? (This was a blog post written by "Jelastic" devs, who make a Java PaaS product).

We seem to be seeing more of this "don't eat your own dogfood" approach to software development lately. The other example that comes to mind is the Play Framework, written in Python and also targeted to Java devs.

Only the CLI utility in Play used Python, and the newest version has replaced this dependency with Scala/SBT instead. I imagine the original reason for using Python for the CLI was because it's pretty damn easy to do that in Python.

Python is great for that sort of thing - but there do exist CLI helper libraries for Java in Apache Commons that make it almost as nice. It's just wierd to me to make development tooling for a langage in a different language - and indeed, if you find yourself doing this you might want to ask yourself why. (Of course there's a clear market reason - the hosted Ruby on Rails market is locked up by Heroku, but the hosted Java market, apart from a subset of JEE supported by GAE, doesn't even exist.)

If you have to ask yourself that question, then you haven't really worked with a scripting language like Python or Ruby.

What java service?

I see what you mean by ironic, but it might be a bit less ironic when you consider the history of the name "JavaScript." Its original market positioning was as a lightweight complement to the heavier Java. Now we have other languages serving in that capacity as well: Python, Ruby, Scala, etc.

I don't know if there will someday be a language that is as easy to work in as a Python or Ruby for simpler programs but which lets you get (almost) as fast and fine-grained as C as your features stabilize and you start to optimize, but while we're waiting for Godot, it's not too ironic to use different tools for different projects, even if the project relates to one language in particular.

(And I don't know why you were downvoted for this. Sheesh, downvoters.... I'm contradicting you a little with my post, but I see no reason why you shouldn't say what you think. So, after contradicting you, I'll now go and cancel your downvote with my upvote. How's that for irony?)

>I don't know if there will someday be a language that is as easy to work in as a Python or Ruby for simpler programs but which lets you get (almost) as fast and fine-grained as C as your features stabilize and you start to optimize

Have you looked in to Cython? It seems like it tries to do what you wrote.

From http://cython.org/

"The Cython language is a superset of the Python language that additionally supports calling C functions and declaring C types on variables and class attributes. This allows the compiler to generate very efficient C code from Cython code. The C code is generated once and then compiles with all major C/C++ compilers in CPython 2.4 and later, including Python 3.x. PyPy support is work in progress and is mostly usable in recent developer versions."

The latest Gevent dev uses libev (same as node.js) I'm curious to know how Cython and the new Gevent will perform vs the alternatives.

First of all I would like to say that I'm no fan of these 54/whatever hour coding rushes, I think it makes people go too fast with poor planning and NO time for testing anything, even for developing looks short, I would rather take my time to plan anything before writing a single line of code.

About Node I have played with it and it looks nice but at the end of the day I just returned to Java and PHP (can't talk about ruby never tried and not sure when I will), why? First, all the Node talk seems to revolve around the hype it generated recently, why I don't know. The idea of the async is nice but stall it with something that takes long to process (and needs to send a response) and you are screwed just like any other language, but I think node may be worse as far as my understanding goes its a single thread dispatching stuff (am I right? please correct me on this if I'm wrong).

Libraries for it, there seem to be too many of the same kind all of them with different ways of working that is not easy, and when I tried it none did what I wanted.

Does anyone know 1 major consumer-facing site that was able to scale with Rails?

I know Twitter kind of had to move away from Rails due to scalability issues.

GameBattles.com also had to back away from Rails after their uptime dropped to 80%.

I am not trying to tarnish Rails, I am just curious to learn about some success stories involving Rails at very large scale.

How about Github?

Shopify is using rails. The thing is rails is bad when there are a whole lot of short life requests. This is what twitter had. The entire rack middleware being loaded for every tweet.

I'd like to know this as well. Some of my Coworkers went to RailsCon a few weeks ago and were remarking how a speaker talked about how they scaled from tens of thousands of hits a month to a couple hundred thousand hits a month. Not very encouraging when that's the sort of traffic we get in an hour and the solution was to use nearly double the amount of web and memcached servers than we do now on our PHP stack.

You can't compare servers to servers. Each app has its own computational needs and caching rules. Your hit is not my hit. One site might simply render a template, another might perform 20 queries and generate giant complex docs.

While it's true you can't do 1 to 1 the fact that my site does ~700 times as much traffic, and it is absolutely the latter of your two types time ten is a cause for caution. This is why I want to hear from some actual enterprise level people that have solved real scaling problems, unfortunately at the conf it seemed like startup size scaling problems were being addressed, and perhaps not very well.

According to this answer from a Twitter employee, they still use Rails: http://www.quora.com/Does-Twitter-still-use-Ruby-on-Rails/an...

Besides that: GitHub, Basecamp, Groupon, Shopify, LivingSocial, YellowPages.com (unless they've changed to something else), etc. Scaling Rails is a solved problem.

Justin.tv frontend is rails, though they may / may not be moving to Django according to Kyle.

http://news.ycombinator.com/item?id=3461476

Of course Rails took YEARS getting to the point of scalability. People are acting like it was scalable on the very first alpha release.

But will JavaScript be able to scale? It has come a long way from the days of onMouseOver but it seems like it's going to be hard unless it gets turned into a different language (i.e. Dart).

What do you even mean by scale in this context?

https://en.wikipedia.org/wiki/V8_%28JavaScript_engine%29

V8 increases performance by compiling JavaScript to native machine code (x86,[2] ARM,[2] or MIPS[3] CPUs), before executing it, versus executing bytecode or interpreting it. Further performance increases are achieved by employing optimization methods such as inline caching.

I mean the language itself, not the performance of the individual implementations. For example, I can't imagine prototype based OO being used for large scale projects.

Hulu, Living Social, All of 37 Signals, Groupon, AirBnb, Scribd, Zendesk, Soundcloud, etc.

Twitter's scale is unlike nearly every single web app online, so I think the real story with Rails and Twitter isn't that "they had to move away from it for scalability reasons," but rather it's amazing that they were table to leverage Rails for as long as they did."

Also, Twitter is more dropping Ruby all together rather than just Rails specifically. Again, this isn't to say that Ruby isn't a great language that works for most people (it let Twitter grow quickly to where they are today), but at their scale with their demands it doesn't work well.

AFAIK Twitter still uses Rails for at least part of their app, perhaps just not for directly serving consumer requests

I'm pretty sure they still use Rails for the HTML application that you see when you visit twitter.com - it's the backend stuff (the data storage layer, the message queues, the API etc) that's been ported over to Scala/Java.

Also who cares if Twitter had to move to another technology. 99.9999% of people will never ever have to face those issues.

Twitter would have hit the wall with any other tech, it just grew like crazy all of a sudden. The fact that it was able to get on its feet so fast though, that's thanks to Rails.

I think this whole "Rails doesn't scale" is the new "Java is slow". Baseless and ridiculous.

Github, Airbnb, Groupon.

Three reasons people think Rails doesn't "scale":

(1) Earlier versions were adopted by applications with anomalous scaling requirements. So, the project had to "learn to scale" on its feet in a very short amount of time. There were predictable hiccups. Or, put even more simply: it's a young framework that got adopted quicker than people expected.

(2) There's an expectation that web platforms can accomodate near-automatic horizontal scaling with maybe just a couple config-file tweaks. That expectation is unrealistic on any platform for which you can easily hire developers. The reality is that scaling to Twitter or even Github's volume is extremely difficult, and the developers who can reliably accomplish it on any platform are extremely hard to find and extremely expensive.

(3) Rails attracts a kind of junior/casual developer that Python just doesn't have, and (weirdly enough) also I think attracts systems developers like myself that aren't steeped in web scaling folklore, which long story short means lots of projects start out by making lots of embarrassing mistakes.

In our practice we see much more Rails today than we do any other framework (there may be some selection bias there, but probably not too much) and for the most part Rails just works. It is definitely not the case that we see lots of startups go out the gate on Rails, go "oh shit we can't scale", and switch to something else.

Thanks, very interesting. Good to know Rails can get there.

> There's an expectation that web platforms can accomodate near-automatic horizontal scaling with maybe just a couple config-file tweaks. That expectation is unrealistic on any platform for which you can easily hire developers.

I can attest to this. I did consulting work for Match.com a few years ago back when they ran around 20 million uniques/month. They are a .NET shop here in Dallas, TX where you can throw a stone and likely hit a .NET developer.

After a few years, their stack looked nothing like a typical ASP.NET stack. It included things like custom database replication, memcached, config pulled out of files and stored in actual code, and lots of other things that went way off the beaten path from standard practices for .NET. New developers often had to learn "The Match Way" before they could be effective. Many things that worked with typical .NET enterprise apps would break down spectacularly when massive scale was thrown at them.

I know Twitter kind of had to move away from Rails due to scalability issues.

Is that true? I know they replaced their Ruby Starling system with a Scala-based solution, but that wasn't Rails to begin with. They also built a new search engine that didn't use Rails. But as much as I can ascertain, the core consumer facing product still does use Rails today.

We're using Rails very happily at Kongregate (with MySQL, memcached, and Redis).

Frameworks don't scale. Architecture does.

From my experience, you can scale a rails app the same way you can scale a django app the same way you can scale a java webapp, and so on. The general problem with frameworks is they try to make you not think about datastore access. Datastore access tends to be the hardest part to scale.

Also, in addition to what others have mentioned, iLike uses (used?) rails - it was a large early facebook application. Not sure what their traffic is like nowadays, but I think it was pretty large in the early days. (Edit: http://blog.ilike.com/ seems to document their growth in registered users all the way back to 2007)

It saddens me that so many developers need to learn why it's a mistake to build a system in node.js right now, just because it's "cool" right now. A variation of this post showed up on HN about 20 times already.

The lesson is simple - don't use node.js to build big apps, it's still immature.

This reads like my life story. Falling in love with a new technology, trying to use it in production, only to find out that it just isn't there yet :)

Node.js will become a great option for web development, but I don't think that node, at this moment, is ready for "mainstream" use.

To me that's the value proposition actually. Invest in learning and collaborating now, then be ahead of the curve in a few years (if interested in getting hired) or be ready to capitalize and know how to hire and make software that only could've been made with it when it eventually reaches some level of maturity. I mean, those that grow it reap the best rewards.

Software is a product of the platform(s) it was built with, in the same way a novel is a reflection of the writer's language more-so than the writer. Microsoft's software is what happens when you use C and .NET; Google with C++, Java, and Python; Facebook with PHP; 37signals with Rails; etc. There are apps that will be built because they could only be built with a unified JS platform. There will be new kinds of software that directly result from the new possibilities of Node, CoffeeScript, Meteor, Firebase, Parse, etc.

Different tools yield fundamentally different results. Once a platform is "mainstream", it's to late, the big opportunities have already passed and the innovation is elsewhere. There's still a chance to be the "DHH of Node"; that spot's obviously already been taken in the Ruby world.

Learning new technologies is something I do, not because I'm interested in getting hired or in becoming the DHH of node. I would do it even if I were by profession a truck driver.

This may sound "romantic", but when your love of technology interferes with you actually being productive and getting things done, ... it's frustrating.

If you are a student, then jumping on a new platform and making a name for yourself is a great thing to do. But I'm an old fart, I was there when Java applets were "the thing that will change the world".

>There will be new kinds of software that directly result from the new possibilities of Node, CoffeeScript, Meteor, Firebase, Parse, etc.

Like what? (not rude, just curious)

As far as I can see there's nothing new under the Sun, so to speak.

I hear you, and can relate to the romance and frustration! I was in grade school when JS was released... but've been writing it for over 10 years now. I tried "everything" else, and it's made be a better JS hacker, but I feel like I'm doing it wrong when I use anything else.

I didn't mean to suggest riding the wave of new, distinct, non-progressive technologies. Their is a progression; it seems clear to me that PG was right, that we (as an industry) are slowly moving toward Lisp. He also said that he felt Lisp and C represent the two "clean, consistent" programming models, and I agree. And that's the reason that JS isn't just another passing wave (although my previous comment did seem to make that suggestion).

JS is something different; it's (1) a good-enough balance of C and Lisp and (2) available on every platform. To me, the situation is clearly that JS will form a solid, durable layer over C. Then the language designers and industry hype machine will shift to langs that compile to JS. This is already happening, naturally, it's just not evenly distributed.

> Like what?

We're still in the early days of compiling to JS, and being able to use it as a modern server-side environment. I don't know what the results will be, but I think the difference will stem mostly from development time. Even though it could be done, in practice you don't end up with the same app if using Fortran, Java, and CoffeeScript, because doing so would take a month versus a week versus a day. To paraphrase Linus from his Tech Talk on Git, speed doesn't mean you do the same thing faster, it changes behavior.

Apps will get written that wouldn't've. I'd argue that Facebook succeeded mostly because of PHP, and in turn, that they'll eventually fail because of it, too. They beat MySpace because MySpace used the MSFT stack; it's like England's victory over Spain due to more nimble war ships. Same thing'll happen to FB unless they evolve when necessary.

This is what Yegge was getting at back in '06 [Dreaming in a Browser Swamp]. He mentioned "Scheme on Skis" and "JavaScript on Jets", which might turn out to be ClojureScript and Express or Railway. Well on our way, and he was entirely right in retrospect (even though a shocking number of smart hackers don't want to accept it and keep on with archaic tech that's becoming rapidly endangered, but that's life).

Light Table is an early example of an app that's happening because of ClojureScript. And many exiting CoffeeScript apps are quite impressive: https://github.com/jashkenas/coffee-script/wiki/In-The-Wild

I think it comes down to being able to focus on design and make fast changes, without needing to worrying as much about the lower layers of abstraction.

I wonder what testing functionality, specifically, they see missing in the Node ecosystem...

I presume it would be something related to capybara, rspec and factory girl.

Looking at the example of integrating those on RSpec's docs page... pretty nice.

https://github.com/rspec/rspec-rails

My guess is a similar Node cocktail would be using Mocha, Tobi, and Should.js together.

I'm writing a server in Node and to date I'm still trying to figure out how to control the main loop from Node. I want to write tests like this:

    1. register some timers and create some sockets
    2. send some stuff over the sockets
    3. wait until the socket handlers received some data which cause the handlers to unregister the sockets and timers
    4. at this point, the main event loop has ended. go to next test and setup a new event loop

How do you do (3) and (4) in Node? I have no idea, and I couldn't find any clear answers on the Internet. I ended up writing my tests in RSpec which launch Node processes, and I just check whether the stuff printed to stdout matches what I expect. RSpec also happens to have a much much nicer syntax than any Javascript testing framework I've encountered.

That sounds like a complicated approach at it. Here's an example of some NodeUnit tests that involve communications with a Redis-based message queue:

https://github.com/bergie/noflo/blob/master/test/MQ_SendMess...

More BDD-ish version of something similar:

https://github.com/nemein/kckupmq/blob/master/test/001_kckup...

Does that test you link actually talk to a live Redis-based message queue that must exist for the test to run?

If so, while that's a nice test there's also a need to be able to fully control the event loop so you can test arbitrary event orders, some of which you may not be able to synthesize (or not easily, or not reliably) but may be able to occur in the real world. Testing event loop code sucks hard enough when you can fake events perfectly correctly, take that away and it sucks even worse.

>there's also a need to be able to fully control the event loop so you can test arbitrary event orders

Hmm, something doesn't sound right there. I'm not really sure why you'd want to "fully control the event loop" for testing, but if you did you can install a callback on every tick: http://nodejs.org/api/process.html#process_process_nexttick_...

If you want to test arbitrary inbound event sequences, you'll need to run at least two processes. You could run two node processes, one generating events for the other. This would probably give you a better test environment then hacking the event loop, too.

"I'm not really sure why you'd want to "fully control the event loop" for testing,"

Because while I'm actually one of the first to tell you that not everything needs to be a unit test and I tend to prefer integration tests over unit tests if given a choice, I will also say that unit tests still have their place, and the way you unit test an event-driven program is to feed the events you specify to an event loop and ensure that the proper actions taken, using whatever is necessary to ensure proper independence.

It's nice to be able to do integration tests, it is not nice to have to do integration tests.

You know, this may explain a great deal of the gap between my opinion of event-based programming and the conventional Node opinion. Testing event-based code isn't necessarily that hard (testing is never really easy in a language where you're always in the local IO monad equivalent, though cracking it up into events does make things challenging, especially if you try to take advantage of closures in your handlers, which unfortunately is advantageous in other ways but hurts testability hard) but it doesn't help that the frameworks never seem to ship with the testing support I just specified and I'm always having to bash it together myself, which tends to show a certain... disconnect in expectations between me and the authors.

#3 I'm not undestanding. Are you doing async? I don't know the socket API but it should be something like this:

it("should wait for socket", function(done) { socket.write("asdfadf", function(err, response) { assert.ok(response.indexOf('token') >= ); done() ); });

#4 If you need a new event loop then write the tests in another module, then simply run your test utility. `mocha` will run all tests in test/*test.js.

RSpec is DOG SLOW! I can run 100s of test in a couple of seconds. Can RSpec even startup in that time?

>RSpec is DOG SLOW! I can run 100s of test in a couple of seconds. Can RSpec even startup in that time?

Yes, of course it can. RSpec is not especially slow. Obviously the ruby interpreter can be slow, for any given rspec test you could write something faster in C or Java or Go or whatever. That's irrelevant.

I'm guessing you're referring to rails tests, where the whole rails stack has to be loaded with each run of the test suite, something that is notoriously slow. There are strategies to avoid this.

Also it is very easy for the naive rails developer to write tests with extremely inefficient usage of the database, which creates horrible bottlenecks.

Check out tap, it has a very simple api that works well for testing asynchronous code: https://github.com/isaacs/node-tap

Here's a tap example that sets up a server and client for dnode: https://github.com/substack/dnode/blob/master/test/single.js

Timers usually aren't necessary with test harnesses like tap or nodeunit since you need to call `t.end()` explicitly or else `t.plan()` the number of tests that are supposed to be run. If a different number of tests actually run or assertions fire after `t.end()` is called, those are reported as errors.

For (4), to run a bunch of test files in a directory you can use the `tap` command. You can even add something like `tap test/*.js` to the scripts.test field of a package.json file and then to run all your tests you can just do `npm test`.

There's also mocha: http://visionmedia.github.com/mocha/

Which has very similar syntax to RSpec.

Try writing tests for controllers with Mocha. It isn't as simple as one might think.

Can you elaborate? I feel that I've written tests for controllers with Mocha several times without much hassle. Maybe you and I are using the word "controller" slightly differently, or are controlling different things?

My apologies I wasn't too specific. I was meaning a controller through express. It's not as straightforward as you would expect with code buried throughout on how to set it up. This is a typical barrier for developers to not test as much.

Express doesn't even have controllers though -- just a routing mechanism.

If you set up your controllers so that they are independent from the routes, then they are just as easy to test.

It could be as simple as: app.get('/users/:id', userController.userPage). You can then test userController.userPage(req, res, next) instead of the route.

True, but in the case of the app I need to start a test server and such. Its those steps that aren't well documented and can cause confusion.

If you think how the whole web development has been changing from server-side generated views to more "produce data and send it over as JSON" style services, Node.js and others start to make a lot more sense. That said, the post has point that on more established platform you can expect to be more productive since you have more established work flows and standard libraries.

Btw. note the terminology: Node.js should not be called "framework", it's a platform. "Framework" is something that runs on platform and you extend/modify it's functionality by writing code. "Library" is something which you call/use from your code.

Since our stack involved MongoDB, it only made sense to live in a JS only environment.

I've never used MongoDB; can someone explain to me why this would seem like an advantage, and/or what it has to do with MongoDB?

Is the MongoDB API simply JavaScript?

I didn't understand that either. MongoDB works very well with many other languages; just recently I was using it with Python, and the driver and API work very well. I guess since MongoDB stores data in BSON, which is the binary version of JSON, the javascript object format, the author decided that JS is closest in spirit to Mongo.

The mongodb shell uses javascript, and stored functions and map/reduce functions and more all use javascript. So if you use mongodb, you'll be using quite a bit of javascript anyways.

Mongodb is a json document store. I use it myself. It works very well with "node" because node of course uses javascript as its language.

But wouldn't it work as equally well with any other language that has a JSON library? With Node you still need to call a function to serialize an object to a JSON string or vice versa.

Not any other language. In JavaScript there is 1 function call to serialize and 1 function call to deserialize. In Java it's much more complicated, for example.

Well in Java, if your JSON library doesn't have a single function to call for deserialize or serialize, it's either a poor library or you should be able to wrap the complicated part in a function that makes it a one-liner. The Jackson library for instance has ObjectMapper.readValue() and ObjectMapper.writeValue() which are both one-liners.

My main point though is that writing in JavaScript doesn't naturally make it easier to work with JSON than any other language with similar functions to serialize/deserialize JSON.

I'm not a Java coder so I'll take your word on it about library support, I've only used the JSONObject, JSONArray, etc. classes. Surely you will concede that JSON being modeled after JavaScript gives the language a little bit of a leg up?

Not really. In theory, it's an advantage in that you can reuse the parser and avoid loading another, but in practice, that's only really important in the browser, not on the server, where the lib will be permanently loaded anyway.

And JSON values map 1:1 to native data structures in most languages - in fact, JSON can be parsed as Python code with the exception of the \/ escape.

It's not JSON, it's BSON. BSON supports more data types, which are needed for a proper database (such as object IDs, longs, doubles, binary data, references, etc.)

You still need a driver and a bson serializer if you use node.js.

Sure. But it's certainly easier than most other languages. It's not a great reason to switch for that, but working with Mongo is fairly trivial in Node.

MongoDB has good drivers for Node (as for most other languages) but also a fully-featured ORM that can save time in these kind of projects: mongoosejs.com

MongoDB also ships with a JS runtime so you can run map/reduce calls written in JavaScript entirely DB-side, as well as other goodies.

So it's fair to say that the language that makes the most sense when using MongoDB is JavaScript, even if that wouldn't be enough IMHO to justify using Node for a project.

I never understood why you needed ORM for a database that stores objects... Validation is the only problem it helps in but there are other ways of doing that.

ORM for Mongo is silly....

It must not be that silly because 10gen hired the author of Mongoose ;-)

A Document Based Store have several advantages over traditional for ex. relational stores ...

Mongo core is c++ and can be utilized from any lingo ...

It is schema less but can be desribed with a schema . it is named collections and on the fly add a tables or rows, no migrations downtime etc.

scary for some ...

Closing into what the oodb concept was setting goals prior ...

Basically they switched for... personal opinion. That's all I can really gleam from the post. You can do it all in Node JS, they just didn't want to.

Screw all the frameworks, I am working with bare-metal Node and loving every minute.

"You can do it all in Node JS"

People like you scare the shit out of me. Golden Hammer is a new concept to you, isn't it?

He actually did lay out the reasons, but your religious zealotry won't let you see it because someone isn't loving your sacred choice of technology.

Lol.. They don't have to love it. They also don't have to write a blog post explaining why they stopped using it.

It was done to generate press, because really no one here ever heard of their app before this. They gained, by slighting Node, and that's why I commented.

definitely agree with this. I can understand why 37signals use RoR, because they built it and know it inside out. But for a newcomer diving into rails now is difficult because the ecosystem is so huge it's hard to navigate, and everything is so integrated and you usually end up bundling a ton of thin that you don't need. With node everything is modularized, it is much more easier to understand what's happening under the hood, and streams are freakin awesome.

Completely agree. I tried to learn rails and gave up partly for this reason. I have been using node for a while now and LOVE it! I really dont understand why people bash node.

In my experience, few people spend much time dissecting technical reasons why node sucks, and most of those people who do don't understand it.

Loads of people laugh at the people who reply to virtually every post with "I use node and LOVE it!", but that's to be expected with any case of over-ebullient advocacy.

I disagree with this. By the ecosystem being huge do you mean the modules and different gems? That in itself is modularized for having the choice not to use them. The availability is what makes it powerful on top of the simplicity to get it all running.

Node is by far not simple for a new developer or new teams running node. Projects tend to drift toward unreadable due to poor design, just try following people's callbacks. I am not saying this is Node's fault but it being young results in developers' lack of experience.

Im sorry but you make it sound like everybody that does rails is a perfect programmer. Rails itself has changed over the years and if you look at an app from years ago im quite sure it will be very different then a current rails app. So again this is just not an appropriate comparison.

I don't think they are perfect programmers. Heck, I use several languages and frameworks, whatever is right for the project. Knowing a language isn't a perfect programmer (we all know that). And to your point, that was the comparison I would make. Node is young so there is a lack of experience with it.

npm is vastly better than the gem package manager imo, making using modules infinitely easier. Furthermore, node modules (the good ones) do one thing and do it right. What I like about node is they give you the core without which you can't live, and you add the layers you want. With RoR you get a huge onion and need to peel a bunch of layers and add others if you want to do things right. I didn't say node was for beginners though. But say for a programmer with a good theoretical background but no web development experience, node is much more appealing and usable.

Do you think Node will be lean and mean forever. It's been around for about two years. Let it grow and it will soon become just as "fat" as Rails and Java frameworks. Funny thing about people reinventing the wheel thinking their approach is just so much smarter. In the end you end up having to face the very same problems Rails guys and Java guys faced in the past. Abstractions, reusable components for X, Y and Z and you end up with something like J2EE or Rails.

>I am huge fan of NodeJs and I believe it’s a very technology and we will see it getting

I think you accidentally word.

Thanks. Fixed now.

Ok. So for a beginner coder looking to build web sites and mobile apps on weekends. They are not looking for a career or job coding. What language?

Assume they know html, CSS, Javascript, SQL. Where do they go and spend their next 12 months?

When do the differences matter?

i think what i'm enjoying most about node.js and the new frameworks, is that they require me to think differently. i'm also optimistic about coding end-to-end in the same language (client, server, database). this is a big learning curve for many folks. i remember when folks transitioned from C to C++ and from C++ to Java -- it takes awhile. i think this is going to be a lot of fun to see where it takes us. also i think work like coffeescript and future IDEs might make this really interesting. Also meteor -- lots of creativity and cool things.

I'd love some elaboration on these points, especially the one about testing as it's an area that interests me.

For anybody wanting to learn about node the NodeUP podcast is excellent.

Reply on news.ycombinator.com