I wonder what the thought process was behind moving their web service stack (partially?) to node.js in the first place. For a company with the scale and resources of Netflix it's not exactly an obvious choice.
Similar is LinkedIn, as an aside -- despite being fairly formidable now, I regularly have entire feeds disappear, their caching is abhorrent, they can't markup text properly, and so on. It seems very amateur hour, yet they regularly publish "how it's done" documents that see wide applause despite often completely contradicting their prior missives.
The net result is a lot of engineers having a lot of time and all these engineers experiment away on technology. In fact, they have quite a bit of NIH syndrome internally because the engineers have nothing to do.
Don't get me wrong, I am not saying that these optimizations are bad. In fact, these engineers measure things properly and then rewrite. It's just that there is no business case for all this time spent. It's like writing web servers in C. Sure, it's possibly the faster than everything else but seriously? Who maintains all this.
> It’s unclear why Express.js chose not to use a constant time data structure like a map to store its handlers.
Well, it was blindingly clear to me and the top commenter in this thread.
I'm not judgmental that it was unclear to the author of this article, after all, I'm unclear on things that should've been stupidly obvious to me all the time. I am totally judgmental that no one on the team proofread the blogpost before publishing it and was like "uh, dude, what would the keys in the map be, regexes?"
But you made me think what I said. I would say this sort of gets into the programmer / engineer debate. Netflix has good progammers like most companies. Good engineers (in my books) are very hard to find. These people are good at being at peace with tradeoffs and understand the requirements of a business at a much deeper level than programmers. Programmers just want to have fun with computers (and there is nothing wrong about that).
Not looking to start any wars, but I was under the impression that if you know what you're doing* node.js is pretty awesome.
This particular bug had to do with a misunderstanding regarding the express API.
* for the most part: understand async and closures/memory leaks.
- "Here we see our latencies drop down to 1 ms and remain there after we deployed our fix." <- Performance (from TFA)
- They can hire from a vastly larger pool of developers. Since they've got both budget and brand recognition, they should have no problems hiring high end JS guys & gals.
> Java is both considerably more speedy for this sort of workload
They're quoting 1ms latencies, that's pretty speedy. Maybe Java could do <1ms and/or support slightly more concurrent requests. How much difference does it actually make when you're already responding after 1 millisecond?
> there's a significantly larger pool of high end Java server developers compared to high end JavaScript server developers
Oh, I would have thought there are far more JS* folks out there than Java.
> Budget and brand recognition are not necessarily in favor of node.js over other stacks. That simply helps with getting the better engineers in general.
That's what I meant - them being in favour of Netflix as an employer - they aren't "Bob's Digital Agency" so they can have their pick of the litter of node devs.
* I'm not sure if the "server" distinction is meaningful here - if you know JS, you know node.js - simply a matter of familiarizing yourself with node-specific APIs.
99% of cases, your bottleneck will be I/O.
In the 1% of cases where your webserver's bottleneck is the CPU, you have much bigger problems than using a hipster ;) language: you're doing it wrong (tm) on an architectural level:
(a) Your processor-intensive/long-running tasks need to be in seperate worker processes and
(b) you need more webserver instances.
I usually try to avoid absolute statements, but I think this may be accurate:
The only CPU-intensive thing your web server code should ever do, is hash passwords.
Can I get a rebuttal to my points along with the downvotes, though?
Is someone disputing that CPU intensive tasks should be moved off the webserver?
Have you personally had experiences where your honest-to-$deity webserver bottleneck was the CPU? I'd be incredibly surprised. In most cases I'd guess you're underprovisioned and/or badly architected.
* This is where we come to somberly discuss interesting things.
1. Your networking pipe into the server is slow.
2. You don't have a reverse proxy in front of the webserver to buffer the slow connections.
3. You don't make decisions about what information to show the user in the webserver; your UI isn't tailored for the particular user whose request you're servicing.
4. You haven't made productivity trade-offs to move work from the developer to the computer.
Google's webservers (for Search) are CPU-bound, and I can guarantee that they are neither underprovisioned nor badly architected, and CPU-intensive tasks are moved off the webserver (the total amount of CPUs for webservers is miniscule compared to the number of CPUs used for the indexing & serving system). But Google has the fastest fiber networks in the world, a massive load-balancing infrastructure, its philosophy is to serve each user only the HTML/JS that they need (to cut latency), it relies heavily on A/B tests & experimentation, and it invests significantly in developer-productivity tools so that all the bookkeeping for the last two points are done by computers rather than humans.
It's not a huge step for Netflix that already has an API approach that is very UI centric as outlined here http://techblog.netflix.com/2013/01/optimizing-netflix-api.h.... To me personally this doesn't seem like a healthy step forward but I hope someone can provide some reasons why you'd want to move to Node.js from, in the case of Netflix, Java.
My preference for node is based primarily on its sane concurrency model. I basically use it as a handy scripting language for doing rapid prototyping of libuv programs. My on-paper plan is to fall back to libuv and C if I find myself really stuck for CPU, but this is something that's never actually happened to me. (I have had to abandon or modify libraries with performance pathologies, though)
There are other libraries with in other languages the same sane model but they tend to be ecosystems within the language community that aren't compatible with the rest of the language. Doing concurrency both correctly and performantly in Java, in particular, is a pain in the ass.
n = 0;
avg = 0;
function twothings() {
thing1(function(ret) { ++n; avg += (ret - avg)/n; });
thing2(function(ret) { ++n; avg += (ret - avg)/n; });
}
In Java, you have access to real threads and can actually run two pieces of code concurrently. This introduces a lot of potential issues, but also allows you to take advantage of multiple cores.
If all you want is non-blocking IO (what node gives you), there are a handful of great libraries that provide that on the JVM.
Node has its perks but for a money making machine that relies solely on being available and providing a good customer experience, not so much.
I can't imagine the ops nightmares at that size, one buggy code path and the entire cluster could be down. These are issues that drove me away from Node to Go, in my opinion Node has way too many issues to run in money-making scenarios.
You can say that about a lot of languages other than node. How many billion dollar companies have their software written in PHP - a language that many people would agree has far more glaring issues than node.
My point, is that I believe your comments miss the larger picture. There is far more involved with deciding which language to build a product with than "which language is best."
I don't disagree with you about Go being a great language - but for most companies, it is not even remotely practical to use. Hiring talented Go developers is hard because there are so few. Their current employees may or may not know Go, what do you do about that? Etc.
If anything the comment on hiring highlights the use of Node being problematic. It's difficult to write robust systems in Node even as a seasoned Node developer.