Thoughts on Rails, Node, and the web apps of today
paulbjensen.co.uk
paulbjensen.co.uk
The whole performance argument is completely beside the point. Yes, we get it, in-process asynchronous parallelism is where its at. But it simply doesn't follow that we need to live in inversion-of-control hell.
Even the simplest node program ends up drowning in error-handling code, because there's no way to automatically propagate errors back to the right place. Every function needs to handle error conditions immediately, because you can't throw an error up the stack, because your stack is completely meaningless.
This is a huge step backward. It feels like coding directly against system calls in C.
It doesn't need to be this way. There's no reason your language runtime can't be smart enough to switch contexts automatically, so that your code gets to be "blocking" even though the actual OS-level process is never blocked. Erlang does this, Stackless Python does this, any Lisp with good old call-with-current-continuation does this.
I don't think there really needs to be a fibers based ecosystem. Isn't it best if modules don't rely on fibers such that they can be reused in either context?
I'm convinced the only way you get this kind of consensus is by building a feature into your language's standard library. This is one big reason that languages with robust "batteries included" standard libraries have been successful. It makes it so much easier to interoperate when everyone can just assume the same abstractions are always available.
Basically it means you can't rely on exceptions at all, so you're back in a "always check for the error code" world.
If you like to have a sync model on node, fine, add a layer like threads-a-gogo and you are ready. By doing so, you give up some of the lightweight-ness and will have increased memory consumption. It is your choice and it depends on your use case. If you expect just a few hundred connections or you have lots of memory, you don't need to think about node.
Raising an exception and dealing with it in the same lexical scope sure is nice.
The author lists some examples, another is: http://www.gevent.org/
Anyway, the call for a sync model is nearly as old as node is and I agree that a solid thread implementation should be part of node for those who like to code this way.
And it is not a call for a sync model. It is a call for a sane API.
About a benchmark, I have none, but it is the nature of threading. Threads need to store and switch contexts. Even if it is just a few 100KBs with 100000 connections this easily multiplies to a gig of additional memory use. In the real world this is often a few MBs per thread making that scale even impossible for a mid class server.
Here's a paper on using userland threads to get the API advantages of threading, with the scaling advantages of event driven programming: http://static.usenix.org/events/hotos03/tech/vonbehren.html
But as far as I know, it's still not competitive. Which makes sense, since the kernel offers far stronger separation guarantees.
We actually got it in the 90s when Tcl had this, only better...
If you prefer that the language does that for you, then that's great. There's enough viable options out there to fit anyone's preferred asynchronous style. Node exposes a lot of the low level elements of asynchronous programming (much like the low level programming you could do in C). You can stick with that, or use a flow control library, or employ any other number of viable strategies to avoid deeply nested callback soup. You get to pick the level of abstraction you work on, and not everyone wants to work on the highest levels of abstraction (where the language or framework handles everything and you just write synchronous style code).
The node community could probably do a better job of directing people to good documentation on those different levels of abstraction (it's a bit silly to assume someone will know to use streams, or monads to clean up the style of their code). When node first started to gain traction, it felt like almost everyone wrote a flow control library. Now the problem is solved in almost every way imaginable. The problem is that new users don't know where to look to find those 3rd party solutions, or how to evaluate the tradeoffs.
You're right to find it maddening. A lot of other people did as well. It doesn't need to be maddening though, and while you have a valid point, it's not especially relevant given the number of possible solutions (you just have to know how to find the style you're after).
The Node ecosystem clearly has no consensus on this point. So you spend a lot of time gluing pieces together.
And I do know how to find the style I'm after. At the moment I'm doing a lot with Q, and I have also used node-fibers. But in either case, you end up doing a lot of plumbing, because you end up depending on other people who chose different (or non-existent) high-level flow control abstractions.
Edit: forget it. Someone else mentioned it in another comment: https://github.com/kriskowal/q/
https://github.com/kriskowal/q/ https://github.com/cujojs/when
But until the Node community can agree on a standard for this, library writers can't make their APIs dependent on these techniques, so we're stuck at the lowest-common-denominator.
And even if everybody was using Q, it's still an awful lot of boilerplate compared to saying the same thing in a language with coroutines.
And there are painful design choices that make it unlikely everyone will ever agree on a promise library. For example: Q guarantees that promises will resolve on a different stack than where they were created. This is nice, it helps you reason about the code. But there are places where you absolutely need to resolve a promise on the same stack (Node's IO handlers, or parts of the window.openDatabase API), and dealing with the edge cases is really gross.
By the way, Q can easily wrap node-style callbacks in a single line.
I disagree. Coding directly against system calls is much harder in C, because you don't have the advantage of closures and flexible typing. But, that aside, node is intended to be a very low-level library that facilitates higher-level extensions and abstractions in userland. It is more like C than it is like Python, and that is by design.
Node is JavaScript on the server. It is not JavaScript-like, or Compiles-to-JavaScript, or Erlang-with-semicolons-and-braces on the server. We don't mess around with the language runtime, we take it mostly as-is, and there is a huge benefit to doing that.
When and if V8 implements generators (as they are likely to do somewhat soon), then I expect we'll see a lot of experimentation in this area in userland modules. They'll have to be run with a --harmony_generators flag, most likely, but they won't need to be compiled or do scary bad-touch things with threads and stacks.
When the area has been explored a bit in those userland modules, and one or a few of them are popular and good and intuitive to use, and V8 moves generators out from behind a flag, and they're fast enough to be used in node without introducing performance regressions, then we'll investigate adding something like this to node-core.
Part of the reason why you meet such backlash from people in the Node community when you complain about "callback hell" is that the model is very simple, and it really is not as bad as it looks at first. JavaScript's bulky "function" keyword does make CPS quite a bit uglier than it is in Scheme, but it's a very reasonable approach to the problem which is extremely extensible.
The crappiest part in my opinion is doing `if (er) return cb(er)` all the damn time. Domains make that a little bit easier, but you're just trading one bit of boilerplate for another, so I don't know. Generators are probably the ideal approach to that problem, but I'm personally not sure they're worth the complexity cost they introduce. I am often wrong, and try to be quick to admit it. We'll see how they change the shape of things once they're a real thing and not just an idea.
In the meantime, use named functions. Use the "async" utility. Use Stream interfaces and .pip() them to one another. And most of all, Don't write big apps! Write small modules that each do one thing, and assemble them into other modules that do a bigger thing. You can't get into callback hell if you don't go there.
That is heart of my complaint. And it's why I made an analogy to system calls, because there you end up doing the same thing -- manually propagating error codes.
> It is more like C than it is like Python, and that is by design.
I agree, which is why the original article simply makes no sense when it presents Node as a competitor to Ruby. They aren't really comparable.
Stuff like "hey @nodejs people lets make a website called http://fibersarestupid.com in which we provide education on how to use callbacks and streams" certainly doesn't help, it just makes node as a community look childish, maybe the site should be called iDontUnderstandFibersThereforeIDismissThem.com... come on.
I might be wrong about it, just not on this bandwagon at all. I still find the true power in the internet is hypertext, and single page apps seem to break it. Of course, I also prefer articles over video and not having a firehose of information thrown at me. Can someone point me to a way to use Node that actually makes sense for something that doesn't have to be near-real-time?
Client side apps are strangely messy, and make it impossible for search engines to index your site. This is exactly why Twitter is largely walking away from their client-side web site.
But no one really qualifies their fanaticism. sure node.js is great! clearly we should use it for our brochure-ware sites too!
So no, we will properly never see that many blogs that are single paged, but it makes sense for your bank application to be single paged.
If you are writing a mail client, or chat app or something, then a single page javascript app makes sense. But I don't think those apps are the majority now, and I don't think they will become the majority any time soon.
I absolutely agree with you. In my opinion the mid-2000s AJAX craze has died down into "AJAX where it's useful, otherwise do whatever." I expect the single page app idea will do the same.
I think the end result will be dedicated pages as the "homepage" of some topic that give full access to that thing and partial access to related concepts. That preserves some of the "I just want to fiddle with the kerblob, don't make me navigate away from what I'm doing" idea without throwing out the hypertext/linking/other beauties that make the internet great.
We went through all the screen mock ups several times before any code was laid down. First thing I knew was it was going to be - for me - a lot quicker to just stick with Rails over node. I played with node before and didn't see my project 'needing' it.
I thought a lot about using Spine. Still hadn't gotten over the Backbone hump, so that was out. And in the end, I just decided that for prototyping and just getting something ready, I needed to stick with the easy and most productive route and just make it a page to page web app. Most of the world still finds this approach acceptable and I was just trying to not get sucked into the technology vortex. 'Real artists ship' I kept telling myself. I did end up using MooTools which seems so old school now, but it totally works fine and I prefer it over jQuery. I felt like a sorta ajax Rails app here and there would be good enough for what I needed to do.
Over the past 8 months, I found pockets of time here and there to put down some code, in spite of a newborn and contracting at a few different startups. I haven't regretted using any of the tech I used and I've been mostly focused on just delivering functionality and getting something ready for people to use. Real artists ship.
Will my Rails app handle 100s of requests a second? Probably not. But, I'm going to love having that problem. It will mean that people will actually be using the app and then I can start thinking about how to scale it up/out.
The shop I'm working with now has a bunch of young kids that are all about node. I think it's great and they even took an old Rails app and rewrote it in node because no one there knew Rails. Ok, dunno if I woulda done that, but it all boils down to - I think - what people are comfortable with.
I'm not that comfortable with node yet. I did see the whole bubbling up error thing and sorta spaghetti code possibilities when i was using it. Like some other posters have said, there are MVC frameworks out there that can alleviate or solve these issues. But at this point I just feel like most apps - most users - will not need or appreciate realtime. I've seen pretty pictures and buttons and ui that have put the sparkle in more people's eyes than if ajax was being used and thusly, realtime.
Is everything going that way (to realtime)? Absolutely. Are we there yet? No way. But I definitely see the writing on wall for Rails as a page to page framework. Maybe they'll come up with something - I've already seen evented php frameworks, so no reason to think this will not get sucked in to Rails somehow. And I've been using Rails as an api platform for years already, so the whole api discussion doesn't surprise me. But for now, I think there's still plenty of life left in Rails to make apps and companies on.
I'd like to see a CMS+Framework like Drupal built in node. Serving an entire dynamic site (say, a corporate site with customer portal) in node could be extremely sweet.
Edit to point out: just because it's in node doesn't mean it's a single-page, ajaxy app. It's obviously the killer application for now, but as time goes on and we link together more and more with external APIs (that might not answer you right away), the async model will make more sense for even a mid-level webapp or customer portal IMO.
One challenge in the Node community is the ecosystem. There are not nearly as many libraries in npm as there are in RubyGems. Of that small set of libraries, a large portion of these have been abandoned. Of that smaller set that haven't been abandoned, there are large portions of missing functionality, are not very stable, or are constantly changing.
There are certain cases that Node works really well for right now, like building real-time chat apps. But having worked on some medium-scale Node projects recently, I constantly find myself re-implementing Rails functionality or rewriting common Ruby gems due to the lack of mature libraries.
Unfortunately there are many other factors involved in building software that matter as well.
Node only becomes useful for me when I need to keep a lot of connections open and handle them simultaneously, and a few other rare situations like that.
As the OP says, Nodejs is not nearly as mature and proven as Rails, but I believe (and that's really a faith-based opinion) that this kind of architectures is the future.
Get your project out the door, if you have to rewrite the down the road, that's fine, at least it's no longer tightly coupled to your whole web app views/controllers.
I just wanted to point it out because I think most people don't know about it, and that's why they are amazed at Node.js's speed (how can a interpreted language like JS be so fast?!).
[1]: It's definitely slower than native code generated by Erlang, I'm not saying it's faster or even on par; but still, it's much better than an interpreter! This StackOverflow answer is relevant: http://stackoverflow.com/a/4220550/347353
Actually does show V8 as being faster than Erlang's HiPE... I guess it pays off to have someone like Google sinking all that work into making things fast.
Not so fast.
That makes sense for highly interactive apps with private content (Trello, Gmail, etc), not so much for public-facing and content-driven sites.
That is, unless you're OK with sacrificing that organic Google traffic to your site?
If not, you have two options: do the entire rendering + biz logic on the server (in which case Rails still makes a lot of sense, though some Node frameworks are getting there), or duplicate your client-side code on the server so that Google's hashbangs work.
The bigger picture is that the web + search engines are kind of broken right now: There's a tremendous opportunity to make progress and use the architecture pointed out by the OP, but as of today there is no simple solution to make the architecture play nicely with Google without duplicating the work on the server (Meteor seems to be making progress there).
Until a simple solution is found, I think we're stuck with the dilemma and will have to continue to do MVC both on the client and on the server.
It sucks.
So I thought he was talking about both. (As I once considered doing, too).
Also very frequently the distinction is rather blurry: Is Twitter an app or a site?
For line-of-business apps or apps in general that can benefit from a rich interface and possibly local storage, full client-side presentation layer (HTML or native) coupled to server-side pure web services can be effective.
At best it's "live" web or "quick update" web.
It just bothers me that the term real-time is being misused a lot nowadays.
I think the point at the end is worth expanding on, though: as long as your API is an API and not a thin connector to a specific backend technology, you can in theory swap out parts or the whole thing. Some time ago my company decided to build a new "web application", and there was some debate over how to structure the communication between (at the time) Flash and the backend. Some argued in favor of a proprietary protocol that makes it very easy for Flash to talk to Java and share data structures. Had we gone with that approach, we would have: 1. not had an easy path to having an "external API" by leveraging the identical internal API already built, 2. been stuck with Flash or at least had a much harder time converting to HTML when it became obviously the right thing to do, and 3. been stuck with Java on the backend and no clean service interface between the backend and fronted models.
It's always necessary to take some shortcuts in development, and I believe it's very important to choose the right shortcuts. Compromising on interfaces and encapsulation is almost always a terrible idea; better to spend the time hiding an ugly implementation behind a clean interface so that you at least have the option to fix it later.
Back then, I was working for a startup that had a product similar in concept to Rails but based on Apache/TCL developed around 1998. The initial architecture ideas were taken from AOL Server, in case anyone still remembers it.
Around 2001 we reached the conclusion that it wasn't scaling any longer for the type of loads that we needed, and more a better language infrastructure was required.
After some research, the framework was ported to .NET, on those days still beta, but their JIT could already yield much more performance than our home grown solution.
Which Apache Tcl thing? Who are you, by the way? Your profile doesn't show anything.
BTW, Rivet can be scaled, like anything else can, although you're right that given the same resources, a compiled language is going to be more efficient.
These guys use Rivet successfully:
- A former developer of Apache Rivet.
I'll be sending you an email describing how it was.
I write single page apps, especially mobile JS apps, BUT rails style frameworks are still incredibly important for 1 huge reason - search engines. If you have a content site, you will have a web front end that is indexible for at least the next 5-10 years. Single page apps are not as indexible/crawlable yet.
So, will you write some single page apps? Yes, are they the one true future, no. The future is probably a combination of native apps on various devices, a traditional web front that is indexable, and probably some single page javascript stuff powering admin panels and highly interactive portions of your site that don't need to be indexable.
For example, in creating ReMeme (http://reme.me) it has a web front that is indexable by google, it has mobile apps and the whole thing is backend API driven by a sinatra app. At this point I have potentially 4 different platforms talking to the api, not just a "single page web app". If it ever gets on more mobile platforms or even on the desktop, that's even more.
The web's a big deal and will stay that way, but you'll probably find yourself writing a bunch of interfaces in different languages, platforms, etc. before you know it. Rails vs. single page isn't the debate. Pick a tool and ship your API, it doesn't matter if it's rails or node or python or php.
Once you ship your API, your bigger problem will be how do you manage the complexity of developing for so many platforms?
A project I toyed with a while ago, but never finished (because I don't need it right now), was an app server that can render and serve Backbone views on the server, for search engines and to speed up the first page load (after which most clients would switch to client-side rendering.) See https://github.com/stdbrouw/backbone-express. https://github.com/developmentseed/bones does something similar. There's a ton of reasons why people would want to stick with Rails or another server-side framework, but searchability isn't necessarily the big problem.
I'm not so sure about this part. To me, it feels like the difference between a web app and an API is simply the fact that an API returns machine readable data (be it JSON, or XML, or whatever). The JSON/XML/etc is just another way of presenting the same data - you should still be able to share a lot of the M and C between your two Vs.
The reason I wrote my jobs in Resque? Confidence. If shit will break - it will be on my side, not inside a package required by another package somewhere deep inside node_modules. I'm reading the source of almost every node package I use. I check all the issues on GitHub before trying to play with it.
While node core is stable, and everything is so damn fast (thanks Redis) and I enjoy every minute of developing with this stack, I find myself too many times inventing the wheel all over again - instead of focusing on my product.
https://workshops.thoughtbot.com/products/1-backbone-js-on-r...
backbone.js sends CRUD requests to rails app which are handled by the controller, and the main page is rails view.
... that said, it's being used with a vast variety of different backends these days: http://backbonejs.org/#examples
Node.js is an ideal candidate for the server, because it is very well suited for building APIs.
I choose the right tool for the right task. Linger on that over-used saying for a moment.
I use Rails when I'm not familiar with the domain enough (this is the business of Software Engineering, we never are as familiar with the domain as the domain experts) and it provides me a platform for radically fast development, in the end I may also have a product that can withhold 2-3 years of evolution, before we need to scale, if even needed (premature optimization..).
I also use Rails when the quality levels are set high, and when I need tons of libraries, and I need those fast. In node, the quality level of such libraries are much worse (if you haven't seen it, you're not doing Node enough time - because I haven't seen it at my first half a year on Node). The number of quality libraries / npm modules are probably several magnitudes of order less than that of Ruby gem world. You're going to need to wait 4 years until you get that level of diversity and quality and it sets up in a comparable way.
And I use Ruby when I don't care about slow clients, or doing system-level work.
At any given time, I keep myself the option to use JRuby, which has comparable to MRI (the "normal", C implementation of Ruby) performance and better on a considerable number of criteria. IMHO DynamicInvoke on the JVM will be a game changer in terms of people considering JRuby against, Groovy, or Scala.
I use node.js when I know the domain will remain small, non-complex, and I'll be dealing with slow clients (real users).
To top all of that, backend systems will NOT turn into thin api servers. Not by any chance. Twitter themselves are rolling back their SPA and bringing back server side template rendering. The number of problem you get into while moving all of your logic and rendering to the client requires a huge blog post which I'll probably make one day. Most people are not aware of that because they never came from full-fledged enterprise level desktop apps -- I used to design CADs and pretty familiar with complex client-side architectures, there are so many dragons there, I'm very happy I now live on the server side, where things are much more predictable.
So no, Rails is not dying and neither is Ruby. With the advent of progress on the JVM and JRuby they never will -- this whole discussion reminds me of "Java is about to die" when Scala came around and "Java is about to die" when Oracle came around. And guess what - it didn't (go read about Java 8).
tldr; I use both, and I'd recommend anyone would, too. Server-side will never be just a thin api, Ruby or Rails will not die (at least not by that sword), and the only thing one might want to work on is a good foundation of intuition of when to use which of those.
@jondot
How complicated was the business logic?
It is already enough that I have to deal with JavaScript on the UI side.
* why the downvote? Is it for being stupid, or for sounding snarky?