Gotchas from Two Years with Node
segment.com
segment.com
If you ran Go as a single thread, you would also run into similar issues. The main advantage of Go is that it makes it easier to parallelize your code thanks to goroutines and channels (a single source file can encapsulate the logic of multiple concurrent processes).
That said, I find that this 'ease of concurrency' makes Go code less readable. In Node.js, it's really easy to identify process boundaries since the child_process module forces you to put code into different files and communicate via loosely coupled IPC channels.
Most of the Node.js vs Go arguments are weak. It's surprising that Node.js is still outpacing Go in popularity in spite of all this slander.
Yes, if you hobbled yourself, you would be hobbled.
The inability to do concurrency properly is not a feature, it's a missing feature. The whole point of node was to be a node.
Having dealt with cooperative multitasking back in the dark ages, I definitely don't believe it is easy. With proper threads, you just write your code in a straightforward manner. With cooperative multitasking, you now have to be continuously imagining performance and sprinkling in otherwise useless calls every time you think something might take a while. When you get that wrong, which will be a fair bit, you have to go back and re-sprinkle. And then when the character of your input changes, you get to re-sprinkle again.
I was ok with it in the dark ages; there wasn't an alternative given the hardware of the time. But now? Even watches are multi-core. I want to use languages that make parallelization easy.
From the perspective of a JVM/.NET/C++ developer I find surprising that Node.js got adopted at all in the server space.
Again, from the perspective of someone that worked in a startup using TCL for server applications, I also don't get it.
Other than being attractive to developers without formal education, and after a certain scale it becomes too costly to re-write.
The work we developed at that company teached me never to use a technology stack without a JIT or AOT compiler for production code.
The amount of money that Facebook has poured into PHP AOT compiler and now JIT is a proof of it, because it is just cheaper to improve the stack than re-write the code.
The "formal education" piece is a little rough though. Count me among the developers working with Node.JS and a university degree.
I think it's more about developers without experience. If you have experience doing something successfully, you are more likely to reach for the same tool set when doing it again. And someone who has done something successfully once is less likely to make naive mistakes when doing again.
I think most issues, like the ones in this article, are much more likely to be attributed to these two facts than to node.js as a language.
Contrived analogy: An experienced builder sets out to build a home. He grabs a Stanley hammer and successfully builds a home. Another person, who has never built anything, grabs a DeWalt hammer and fails miserably. Does this mean that only Stanley hammers are appropriate for building a house?
I don't have specifics to quote but enterprise dev is definitely not a "tiny" percentage
I may be terribly confused here, but I'm pretty sure this is the precise opposite of the facts. In terms of jobs, all the OSS languages TOGETHER are not as popular as Java alone, never mind adding in C#/.NET. It's a common misunderstanding, but one that seems unlikely to do a young coder any good...
Thats a rather defensive position for node in what is a very rare use case for the language. It's unsurprising Node.js is still outpacing Go, given the large number of JS developers and the fact that Go is pretty much worthless for hosting front end web applications (you won't find your favorite asset pipeline in Go).
Its not surprising at all that they switched to a different language for data processing & pipelines, and its still somewhat surprising as that they chose Go, given that most teams in a situation like this would switch to the even more popular JVM/Spark/Storm/Kafka stack.
Finally, your statement that The real underlying issue here is that one CPU core is given too much work while others are more or less idle. isn't accurate - the issue is one thread was has too much work - no modern OS built in the last 20 years would allow a single process to hog all the CPU time unless you explicitly turned off the kernel's scheduling. The root of the eventloop issue, is eventloop specific and its even more Node-specific since the event loop is pretty much the only way to achieve concurrency in node. Other languages (like Go and Java) at least have options for other models of concurrency.
Consider the following - what if both processes got tied up? Do you just start another process? Would it feasible or wise to run 1000 processes (no it wont)? However this is a problem that you won't come across in Go by using goroutines and taking advantage of its scheduler, as you can easily run 1000s of goroutines performantly.
That said - this is a rather narrow use case to make the judgement that one language is better than the other - its just the case that Go is likely better suited for these kind of services.
C makes you do array out of bounds checks. Javascript makes you worry about tying up your event loop with eg massive string processing.
Just get a friggin asynchronous JSON parser if you're running it on untrusted client input (ie any client input). It's not that hard.
Maybe node should provide a "tainted" feature for modules to mark variables that are "untrusted" and provide some warnings when functions like JSON.parse are run on them.
The upside of JS is massive - easier to reason about control flow than threads, and much easier to build something much FASTER and efficient than threads.
The problem for me with Node here is that whole cooperative-multitasking thing doesn't directly buy you anything. It's a historical accident, not a necessary downside of an otherwise-positive choice. That's distinct from a browser or a GUI environment, where letting a single thread control the display and events really does buy you things you care about.
The underlying issue is that when you have a consumer-facing API which accepts HTTP requests with a body, the first thing you should think about is limits.
> Consider the following - what if both processes got tied up? Do you just start another process? Would it feasible or wise to run 1000 processes (no it wont)? However this is a problem that you won't come across in Go by using goroutines and taking advantage of its scheduler, as you can easily run 1000s of goroutines performantly.
I have no experience of go, but my understanding is that goroutines are green threads multiplexed over a small thread pool. If you get 5 MB of JSON in N different requests (N=number of cores) at the same time, I don't see go generating free CPU time out of thin air. The usual way to go about these things in a language without multithreading is to have a queue and a process pool, but this also won't magically solve the issue if all cores are busy.
You don't, but the scheduler normally won't allow one thread to completely starve the cpu. Of course, its clear they should be using limits, however JVM, glibc threads scheduler or Go's green threads likely wouldn't allow a single thread to completely starve the CPU, eventually the scheduler will step in and divert resources to another thread.
Without limits in a threaded solution, you would see the latency increase, but you wouldn't see the application stop taking requests altogether.
However there are real benefits for having an event loop concurrency, so this shouldn't be taken as a reason one model is strictly better than another.
Err no you wouldn't,you don't have to deal with interprocess communication when you deal with Go routines, you share memory between goroutines ie , true concurrency where as in node you'd fork some stuff, serialize data between processes and message with redis to make sure everything is notified something happen. Node isn't concurrent. Go is.
That's why you read all these blog posts about "how I moved from Nodejs to Go(or rust)" and that's why you'll read more of them.
I find that approach troubling. Process boundaries are expensive, because you have to serialize and everything each time you cross a boundary.
I also haven't used Go, but I think you can get great clarity with something like Akka's Actor model. And to do it you don't have to pay a large serialization tax until you move particular actors to other machines.
Sometimes you want to scale up to the box you are on and the most efficient way to do that is to use threads in a single process.
When you hit the limits of a single machine then it's time to start scaling out to multitple machines. You are right that then you have to start paying the communication/serialization costs and in return you get much greater scale.
However you don't have to start paying that cost until scale in that direction. You can get quite a lot out of multicore machine these days without having to pay serialization costs if you use threads.
Many times the serialization costs aren't worth the benefit unless you are getting a whole other box with another 32+ cores out of the deal. Paying it before you get that benefit isn't efficient engineering. And for some people choosing a language or framework that forces them to pay that cost before it's necessary is a bad idea.
Well, that and Go is not bounded by a VM (jit or no), and will simply be faster than JavaScript, given the exact same logic. Don't discount the cost of running on a VM, and all of the abstractions which are thrown on top of plain JavaScript to help manage the callback complexity.
Given time, don't assume any particular language implementation will always be faster. Go might only run faster than Javascript on the odd years, depending on corporate budgets for compiler / VM tuning the previous year.
It took us a year to arrive at the solutions above, and many prominent members of the community scoffed at a number of them. Some are still scoffed at.
For example, everyone still thinks that promises should not catch errors, as if somehow throwing on typos is useful. Its not. The real solution here is a type system, the sensible error capturing model of promises that doesn't destroy all assumptions about code (unlike domains), and a sensible library like bluebird that reports rather than swallows up unhandled errors.
People also swore that streams are the best thing ever but the reality is that the built in streams are an organically grown design that accreted many flaws along the way. Most of them can be mended though and with streams 3 things are finally starting to be acceptable.
Great library and the API offers a lot of conveniences without being overwhelming.
What have you found sufficient for this?
Some of them can be found in https://github.com/spion/promise-streams - the others we haven't extracted to an npm module yet. (Promise streams also has a minimal extension of built in streams to make them work better with promises).
We never managed to find a full replacement though. Probably not worth the effort either given that it will always have to wrap existing streams, and the only drawback of external functions is that you cannot invoke them as methods. Although, one contender that I have hopes for is WHATWG streams [1].
note: to replace event emitters we built https://github.com/doxout/promise-observer which gives you a lot more power and control in terms of execution order and goes away with the "multiple events per object" stringy design that makes things harder for typesystems like typescript/flow.
Apologies for taking this opportunity to shamelessly plug my streaming JSON parser, which I recently open-sourced here:
http://www.adam-bien.com/roller/abien/entry/a_java_ee_startu...
Everything except that special sauce called green threads (or goroutines). They are quite popular these days for good reason - callbacks suck.
BTW, I don't like the term green threads because they have a connotation of being scheduled onto a single OS threads, while fibers/lightweight threads employ parallelism.
I mean, there's plenty of good stuff too, but... let's just not pretend that it's a choice without downsides, shall we?
Ironically, Java actually ends up being one of the better languages for that; there end up being a lot of gaps in some of the others. Just a $0.02 example: Ruby tools like Nori and Crack that are designed to make idiomatic Ruby structures out of XML but have fundamental flaws in how they handle niceties like namespaces. (Nori can strip them out, or you can leave them in place as the short-name string, but you can't normalize a given namespace URI to a given string, so good luck if some tool that was sending you xmlns:ns0, xmlns:ns1, xmlns:ns2, etc starts messing up the order. I'd consider making a pull request, but chose to leave tech debt in that part of the stack instead.)
> Yes, if you choose a crufty framework like J2EE
I do believe that using J2EE was exactly the suggestion of the GP, unless there is a subtlety in the usage that I got wrong (I understand the 2 has been dropped in later EEs). And are the ones you suggest "tried, proven, standardized" like I was pitched?
And the cruftiness of the standardized framework is the even-more-fundamental answer to the original question of why-not-Java: it's not strictly because you can't do things with it well or quickly, it's because it has a crufty reputation.
I would say you would have to have really extreme requirements to handle those type of things yourself.
Really? I'm just a beginner with node.js, and I've been deeply frustrated by error handling, but if this is true, that's pretty damning. In just about every other web framework under the sun, you can go wild with exceptions and the worst you'll get is a 500 response for that request. (Yes, worse behavior is possible but very uncommon.)
There are web server packages on node that do catch errors that happen in synchronous code. For async, you can use promises, which take errors into consideration.
I'm not sure what happens on unhandled async errors on other languages, I guess in Java you could have a dangling request (happened to me before) or it could crash the app (also happened to me before). C++? I haven't really done much webserver work on that one so someone else may be able to write more about it.
Also obviously PHP, you just get a white page or actual full error stacks sent to the client, the joy! (edit: of course, if you don't handle it properly)
Also bonus points if the event emitter object is private to some module that doesn't expose it and doesn't attach an "error" event handler to it.
This not the same as other technologies. PHP will return an error but will not take your site down as node does.
We are currently in a world where we have a huge amount of traffic on almost every web app with just a discrete success, but we still make the mistake to pick a technology that seems "good enough", instead to pick a "great one" because looks slightly harder to manage/learn/deploy. I know that during the early stage, pace is very important and RAILS or Node are easier and faster to handle compare to Scala or Erlang, but sometimes another technology at the beginning would save a lot of headaches and night calls. We still fail at the very early stage to choice the right technology, but nobody is afraid to admit it and to switch, I find this amazing.
I don't see anything wrong with planning a rewrite X months into a product, since 90% of things don't make it to month X.
As an example, I recently migrated a Node app from MongoDB to Postgres. This ended up taking two and a half weeks, due to re-writing a fair portion of the server-side code. That's a long time to go without delivering new features or fixes. We justified it because we had inexplicable data loss (not pinpointed on MongoDB, but a poor reputation is a hard thing to remedy) and our data model did not suit a document store. But you have to then accept it when the business folks say "well, why didn't you get it right the first time? Aren't you supposed to be the expert?".
As technologists, of course we find it fun to try new technologies. But outside of the main tech hubs, a large proportion of developers aren't working for tech companies whose main product consists of web services/APIs, in which case we need to always consider the costs/benefits of any tech switch. If it's not justified, you're stuck supporting flakey apps until you can move on to the next gig / learning experience.
A technology switch can have multiple sides, sometimes the success is way beyond expectations and the current implementation doesn't fit the real requirements, making the switch looking more to a success rather than a failure. On the opposite it's very common to have the exact same problem you described, new trending technology picked, hit the limit, switch to and old more robust solution, this is definitely a failure and is something managers don't like.
I faced a similar problem with Mongo 2 years ago, had the same switch to PostgreSQL.
Now should have Twitter spent the first 6 months of their life building out the perfect infrastructure with proven tools spending the little money they had mainly on engineering or were the justified in pushing that technical debt down the road to focus on other things today.
It seems to me, for most startups, that the marginal cost of building it "right" today is much higher than rewriting when you can/if you need to.
For example, I would argue that going a couple of weeks without delivering new features is far from a long time. If your features are so simple that they can all be added in that sort of time frame, and if failing to do so is enough for a competitor to cause serious damage to your business, then it seems unlikely that you had a strong business model/value proposition in the first place.
Likewise the idea of sitting on a known and unexplained data loss bug indefinitely is horrifying. Again, if you don't consider removing that kind of liability a priority, it seems like a matter of time before something disastrous happens. Depending on the nature of your work and the data involved, this might even amount to negligence and give legitimate grounds for affected customers or regulators to take legal action.
Finally, if you have management who expect everyone technical to be an expert on all technical tools and make perfect choices, then they are both ignorant of how techincal fields work and extremely poor at managing risk on a project. Once again, with that kind of person at the helm, you are already doomed.
Personally I favour tried-and-tested over new-and-shiny for most projects. My experience has been that many new and trendy tools have a good sales pitch but don't stand the test of time. After using them for real for a while, developers often start to understand why things were done the way they were done before and discover that this week's silver bullet comes with limitations or risks of its own.
In any case, whether you're using time-tested tools or expecting that newer really is better, building up technical debt to unmanageable levels will kill any software project. You learn as you go along, and at some stage your greater experience may suggest that a different approach would give much better results, and then you have a cost/benefit question of whether and when it's worth doing that work.
On the flip side, over-engineering before launch means time spent NOT MAKING ANY MONEY because you haven't launched yet. It means pouring engineering effort into something whose success is still hypothetical.
If the biz folks want to know "why didn't you get it right the first time", ask them why they didn't become millionaires at their first jobs.
Also, once the business gets hold of a prototype, it can sometimes be hard to convince them to pay to replace the prototype with a new codebase which seemingly does exactly the same thing; after all, what they've got works, right? (For some definition of "works".)
It can also be difficult to work out exactly what needs to be kept from the prototype; how do you tell what functionality is intended, and what is just a non-essential byproduct of the implementation? Yes, you can formalise the specification, but this takes time, and customers will complain about any changes to functionality.
You'll be served in good stead if you pick a tech that can support all the normal boring stuff that a robust web app needs, while also providing rapid development capabilities.
Timely and relevant quote: https://twitter.com/patio11/status/587769019261829120
People uses Node, Rails or whatever because it's easier to develop with. Then you understand your problem and optimize them away from your core.
I don't think this way of handling thing will ever change.
TJ Holowaychuk desertion was a strong hit on Node troops' morale.
Did anything of this change after IO.js was forked?
Suffice to say his exit hasn't changed anything, his efforts have been taken over by others and NPM is still growing at a fast rate.
That all successful apps have a lot of traffic doesn't matter, because most apps are not successful. Building for scale from day 1 wastes resources better spent on making something people actually want, because that's what increases the chance of getting to where scale actually matters.
I think the real trick people should learn is to stop building monoliths, so that when item X is problematic it can be easily swapped out. But even that only makes sense if the cost of a more modular approach is relatively low. It would be great to see good early-stage toolkits that encourage novices to build more modularly.
I don't know about Scala, but if one were to start from tabula rasa, I'd argue Erlang is easier to handle than Node. You have a standard set of patterns embedded in OTP and a well-defined process model purveying the entire language, whereas Node at its core being a reactor-based event loop means that you're bombarded by a variety of concurrency patterns that are all lacking in some way. This is besides all the advantages of location transparent distributed nodes, the Eshell in general, multi-core scaling (modulo Amdahl) and a pattern matching engine like none else.
Rails isn't an adequate comparison, as that's a framework. Nitrogen/N20 + BossDB or Chicago Boss would be close competitors.
I have commit access to Chicago Boss, and while it's a really cool effort, and is well suited to some niches like the one I'm using it for, it is very, very far away from being a competitor in terms of the flexibility and oodles of gems you get out of the box.
I don't know about "flexibility". It's much less rigid and opinionated than Rails, and it's easier to maintain modified versions of the source code in your project due to how Rebar handles dependencies. So I'd say it's pretty flexible.
Default library support, yes. But it's not that bad. Most major tasks are covered. The module system and resulting encapsulation (on top of runtime + OTP guarantees) means that using dumped libraries is easier and more reliable, so I had no qualms with reading and integrating, e.g. a wrapper to GraphicsMagick that serializes output to native Erlang proplists for batch image uploading.
The ETS session engine, the not strictly OO data mapper, functional tests, inbound mail server, in-memory MQ and model event watchers plus first-class WebSocket gen_servers are all nice perks.
My main concern is that commit activity has been dwindling down and the present lead maintainer (danikp, I think?) seems to be only sporadically active. But it can still be salvaged, I'd wager most of the serious users have private forks.
EDIT: By the way, out of curiosity, what "niches" are you using it for?
But there is a lot of potential in the future. ES6 greatly improves the Javascript language and it's possible ES7 or 8 will add types. If something like TypeScript becomes standard Javascript and async/await is added, the language will become a "serious" language for many developers.
I think the Node of the future will look nothing like the Node of today. When Javascript/Node gains these features, it has the potential to become a true server side language.
1. Event loop. Yep, always be careful not to block it. But, I think this is more of a tuning thing. If you're using Java, you figure out how many requests you can reasonably handle and make that your water mark. Do the same thing in node, if you take too many connections, start refusing them fast and not queuing them forever.
2. Exceptions. Honestly this has never been an issue. We've used Promises for async flow from the beginning. This means that there is nearly zero code that is not inside of a promise chain and hence inside of a try {} catch {}. We just don't have this problem and we don't crash.
3 / 4. Oddly enough, I found streams to be a solution to this problem and not a cause, if using back pressure properly. I highly recommend highland.js.
This seems to become a common pattern... start with Node, hit some limits, rewrite microservices in Go.
…such as Haskell or Erlang.
Its the new shiny thing.
EDIT: Very much so, they were actually released to the public in 2009.
Sometimes the shiny newness is in the public eye rather than absolute terms.
It could also just be random chance. People and communities latch onto things for all sorts of reasons.
But a few years ago I effectively had to start re-learning Java from scratch (I knew the language but the whole ecosystem had moved on from when I first learned it), and it certainly didn't take months to get productive.
> But here… there be dragons.
That's one of the things Erlang solves pretty well. Sometimes I sort of envision it as that robot dog/mule thing that they keep kicking and it gets back up and keeps going.
https://www.youtube.com/watch?v=cNZPRsrwumQ if you have never seen it before.
http://morepypy.blogspot.de/2014/11/tornado-without-gil-on-p...
A much better way to do this would be to have a very lightweight API putting events into a stream (Kafka or, since they seems to be on AWS, Kinesis [1]). Let the stream absorb that crazy customer data, and let your data processing run full-speed from the stream. You'll get blips and slow downs, but it won't affect your ability to receive more data. Log out any errors or malformed data so that customers can see the problems. Do your profiling and optimisation, but avoid losing data.
[1] We're using Kinesis. Very easy to provision, does what it says on the box, and can easily handle thousands of requests per second.
Let the Kafka project handle the nuances of durable, performant queueing - that's not your business model.
Do the parsing in a web worker.
PHP: Starts reading from database, blocks everything else, returns data.
Node: Starts reading from database, lets it go and continues with other stuff until the data is ready and then returns it.
I understand it in theory but since Node is single-threaded, doesn't it need to use that thread for the database operations? Which means it's blocking the program until it's done anyway?
There are some very obvious architectural issues here, issues that aren't tied to the choice of Node.js. I wouldn't want to use a provider whose stack was this immature, to be perfectly blunt.
For example, an API shouldn't do any of the processing they describe. The API's role is to handle submitted requests as fast as possible, so it must off-load the actual work to a queue that can be processed elsewhere.
I'm also skeptical of solutions like YAL, at least the way it seems to be used here. IMHO you'll want local spooling via something like rsyslog, so that you're impervious to network failures. Logging directly to a remote server means you become completely reliant on that server.
"Node is good when you're using a linter, when you're writing your back end in go, when you write MASSIVE unit tests..." May be it could be much simpler to use a more suitable langage, no ? Just saying...