Farewell Node.js
medium.com
medium.com
He has contributed a lot to the nodejs ecosystem (around 600 packages! [1]), and it saddens me to see him go. The Go guys should be very excited to have him on board!
Also listed on his GitHub - https://github.com/visionmedia
He's bored, and moving on. That's it. He's trying to justify what must be a painful decision by making a bunch of abstract claims about how Node isn't a production-ready language. The elephant in the room: how is it that TJ was able to build so many things, over so many years, that are used -- in production -- in so many places, in a language that is "difficult to debug, refactor and develop."? Is he a masochist?
I think he's hit a personal wall. It seems he has a beef with the way the core is being developed. And Go is indeed fun, and exciting. This isn't surprising. People get burned out.
Like I mentioned in my post, I decided to rewrite an application that I had been working on for the last month with Go. I decided that if I could do it in a week, and if it went well that I would ditch node. It went even better than I had hoped, and is much less frail. Node has many design flaws down to its very roots.
I'm not saying those are due to terrible engineering or anything, but they were choices that were made and Node is stuck with them for the foreseeable future until breaking changes can be made. That is if the core people can admit that it's not great to work with :)
I think the main problem is often people working on the depths of some system forget to really use it. It's so easy to get caught up in the details but if you don't step back and use your own product you're screwed!
I wonder about this "enjoyable" criteria. I'm assuming you're not claiming to be the only person who has written large Node systems, or that nobody has ever written larger, more highly trafficked, more complex Node applications than you have. So the point seems to be: for you, Go is more fun than Node at scale. As a well known contributor, that's a valuable perspective. I'd like to know more about how your personal experience could be translated into general truths all developers could understand.
It seems to me that Node is used rather extensively by the core contributors. I don't get the impression that they have never actually used the product, based on their repos etc.
Thanks for all your work. I'm sure the Go community will similarly benefit from your efforts.
I'm definitely not the only one writing large systems, but I think the tolerance level varies per developer. Also plenty of people make a living/name from Node, of course they're more likely to praise it than to be honest about its faults, I'm definitely a minority in that respect. Having broken concepts is one thing, but resisting change when they're obviously broken is not a good thing. Many of these same people make money from consultation, where it's advantageous to have a broken system. I'm not trying to screw them over but someone has to be honest.
It took years to get .addHeader() in because no one in core believed in a progressive API, they didn't use node in real-world applications to see the need. This still happens all the time, take npm-www for example, Isaac is a rad guy but him aswell as most other "core" community members advocated building tiny little things and stitching them all together, and just recently realized that in practice this doesn't scale, thus ended up going with Hapi. This lack of insight is all over Node as a community.
It's hard to describe well, but I hope people will try Go (or similar alternative), you'll really see how much more robust it is. If node fixed its conceptually flawed event system, rewrote http so it wasn't awful to work with, and fixed streams then we'd have a pretty good system to work with. It's very easy to pass off problems with node as problems you'd have with any platform, but that's unfortunately just not reality.
“I think the issue is we've been slowly accepting that the node is the right way to do things. Five years of working with someone will do that to you.”
Who is “we” and what is the evidence for the conclusion? How is it true that 1) In 5 years all coworkers think exactly the same way; and 2) Some purported truth about human behavior relates to the general usage of a programming language within an opinionated and free community? This seems like some random statement without any (provable) basis in reality.
“It has always bothered me that people would advocate such broken systems (streams etc) when real-world alternatives were better even before now”
You’ve started companies, joined companies, and encouraged Node technology at companies. You’ve advocated for others to use the systems that you’ve built. These companies have investors and other who trust your judgement. Why did you advocate a broken system, for so many years? Or did you come to this conclusion just a few weeks ago? What led you to that conclusion?
“I'm definitely not the only one writing large systems, but I think the tolerance level varies per developer.”
I’m not sure that anyone I know who builds massive enterprise systems ignores the “bad parts” of a system. None of them are tolerant of bad ideas. You really can’t be if you’re a professional with responsibilities to your team, your company, and shareholders. To suggest that Node is popular because Node developers are more tolerant of bad systems is, well, a weak theory that I think doesn’t stand up to reason.
“Also plenty of people make a living/name from Node, of course they're more likely to praise it than to be honest about its faults, I'm definitely a minority in that respect…Many of these same people make money from consultation, where it's advantageous to have a broken system. I'm not trying to screw them over but someone has to be honest.”
It’s not controversial to point out that you have advocated for (praised) a system that you believed was full of faults, for years, for great profit, in many senses. So you’re indeed a minority, but not in the way that you think you are. All of this sounds, to me, a little screwy.
“This lack of insight is all over Node as a community.”
I’ve re-read the paragraph preceding the above conclusion, but have failed to find a cogent argument. Something about #addHeader, and Isaacs is rad, and something else about “building tiny little things and stitching them all together” is a fools errand — this seems to be an “insight”. From Eric Raymond’s Unix rules: “Rule of Composition: Developers should write programs that can communicate easily with other programs. This rule aims to allow developers to break down projects into small, simple programs rather than overly complex monolithic programs.”. Unix has scaled pretty nicely.
Now, you may be right. Also, Raymond might be right. Also, the entirety of the thousands of developers who build serious, professional software by following this credo might be right. Or some may be wrong. Or all may be wrong. But the point is… I still don’t see you producing an irrefutable argument, especially since you want to conclude that Node is fundamentally broken, isn’t “ready for prime time”, and so forth. I mean, you’re the most prolific contributor to the Node project, and you have advocated on several occasions for exactly this sort of methodology. It’s confusing.
“It's very easy to pass off problems with node as problems you'd have with any platform, but that's unfortunately just not reality.”
Not sure what you’re saying here. I could link a few hundred articles covering every language, including Go, that say the same thing you’re saying about Node — but I don’t know what they’re getting at either. No programming language solves all problems, and is easier than all others, and has no mistakes in it, and etc. Do you dispute that? If not…why exactly is the fact that Node has some problems, and some imperfections, and some mistakes, a reason to dismiss it outright? From your earlier post: “my claims of node not being production ready are 100% legit, and based on real-world applications.”. 100% legit? That’s pretty legit! But howzit legit?
I’ve been a little hard here, but I feel I had to be. Again, you’ve been a great contributor to the project, and I hope you do some more great things.
At StartHQ (https://starthq.com) we've abstracted away all async code with a sync layer using fibers and that has served us well over the past year, but I can totally see how trying to mix the two would have been a recipe for disaster.
Here is a video of a talk we gave on this: http://www.youtube.com/watch?v=pmyDJnEza6A and here are the slides: http://www.slideshare.net/olegp/start-hq-server-side
I don't disagree with the gist of what you're saying, and I think you're correct. But the premise of Node being high-performance is perhaps more reputation than actual fact. I think it gets that reputation because it is higher-performance than some more popular options, but in the big scheme of things, it's but a medium-performance platform [1].
[1] http://www.techempower.com/benchmarks/#section=data-r9&hw=i7...
http://www.slideshare.net/olegp/server-side-javascript-going... - there are some graphs from slide 46 onwards.
The big issue with generators is making sense of stack traces while debugging. And due to the nature of how generators work we'll probably need an fairly creative solution here. I'm fine with generators today, but it would be better if await sugar landed sooner.
- It is outside the JS specification
- Needs to be updated for newer versions of node. Last commit says "Support for Node 0.11.10". Also, it needs to keep up specifically for every platform.
- Will never work in the browser, unlike yield which you can get working on current browsers using transpilers. Increasingly important, as we see more code sharing between client and server (isomorphic JS).
If you ask me, fibers are quite risky.
It works fine with the latest Node for me. Fibers relies on libcoro which has only had minor updates over the past four years.
I'm not sure where you're seeing this code sharing. In practice the only code that is shared is utility methods that don't do any I/O, so whether you use fibers or not makes no difference.
In fact, Meteor, being the framework that is pushing the boundaries of code sharing between client and server, uses fibers on the server.
My co-founder alexlamsl and I are actually working on something akin to Scala actors for JS that is portable between the browser via Web Workers and the server using Node and Common Node (https://github.com/olegp/common-node). We'll blog about it at https://starthq.com/blog/ in the near future.
He tried to do a lot many things single-handedly. Which is why he needs a lot of maintainers. I don't blame him for overloading and stressing out. That is the way how it goes in node.js. Working in node.js is like battling a multi-headed beast. The core javascript keeps evolving, node.js isn't even in version 1.0, API changes a lot. There are lot of blanks to fill and too much time goes into boilerplate stuff and managing existing code.
It is just not rewarding and fulfilling to contribute to node.js. On top of that he faced a lot of friction from other contributors. He had some really difficult times, like express-connect conflict, bower-components conflict. Having handled conflicts too many, some people also consider him as being rude and too self-centered. Besides he is young and is probably one of the first things he has done he is serious about.
Many times before I asked to myself how does this guy do it. In a community where to contribute even a little one has to put in a lot of effort, his contributions are gigantic. I hope he continues to work, although not at too many projects . His experiences will definitely be handy.
The core node community was very unreceptive to change since the very beginning, to be honest Ryan and Isaac were some of the few humble people in the bunch. I've always been a "threat" to "the node way" of doing things, whatever that means. The elitism gets a little old.
running it today tells us that he's still number one:
rank percent packages author
---- ------- -------- ------
1 0.60 % 548 tjholowaychuk
2 0.44 % 401 substack
3 0.41 % 379 jongleberry
4 0.40 % 367 dominictarr
5 0.37 % 341 jonschlinkert
6 0.37 % 337 sindresorhus
7 0.36 % 331 juliangruber
8 0.34 % 317 mikolalysenko
9 0.31 % 289 raynos
10 0.28 % 260 mathias
11 0.28 % 257 hughsk
12 0.28 % 255 forbeslindesay
13 0.26 % 239 tootallnate
14 0.23 % 216 clewfirst
15 0.23 % 208 azerSame here, but Go has productive libraries that can help you hit the ground running. Just with the packaged standard libraries, you can do things like have a useful webserver going. I prefer Rust though and might stick with JS until I exhaust it .. or until Rust matures enough to put into production and then jump fully on to Rust. Rust compiles to llvm-bitcode, unlike Go, so that's a win for me.
edit: I stand corrected - https://github.com/go-llvm/llgo
We use it at work for both packaging and deployment and it's awesome!
TJ has done quite a bit to advance the state of the art in JavaScript development, it will be interesting to see what he does with Go.
http://www.quora.com/TJ-Holowaychuk-1/How-is-TJ-Holowaychuk-...
Additionally we are now past Rails wave, Node.js wave and into Go wave.
And the narrative is similar as well: starting a new project with the shiny new thing is cool but maintaining it is boring and awful and to quote the OP "I need maintainers!".
Part of the problem is the language of course. Maintaining even a medium sized codebase written in a dynamic language is a challenge. But the other part is the mantra that "coding is easy" and "everyone should do it". And now there's an entire generation of developers whose idea of software development is "writing an app in Node in a weekend". Which of course doesn't require years of maintenance, complicated tooling or even adherence to common best practice.
Now, I've heard he works for Segement.io (in SOMA), but does so remotely.
Have he said anything about that in other blog? Or anyone of you have sent him email about that?
I use node.js (0.3 to now) quite a long time. I also feel the smell of node.js, forever, express and socket.io. Recently, I am considering switch to Elixir. But I am afraid now.
From day one in node.js to now, we know that node.js is not prefect and not fastest. But it is so popular and some many great guys, like TJ, contributed many packages into the eco-system.
Go is not prefect too. But once it get more popular and more packages. Better vm like Erlang or X or Y may be just ignored by devs.
Hey! Mostly Go because it suits what I pictures as my ideal language pretty well. It's simple, C-like, great concurrency primitives, great standard library. I've had bad experienced with anything JVM so I'd stay away from Clojure, and Erlang has some legacy baggage but I'd still like to give it a better look some day! Elixir is cool but too Ruby for me
I wish more people realized this. Generators seem to be billed (ironically by TJ amongst others with his Koa project) as the solution to callback hell but, having tried Go, I got the same felling that actors are much easier to reason about and debug (although concurrency is still hard).
On the client side, web workers allow parallelism but the API is inconvenient to use: you need to have a file for each web worker whereas Go's `go` routines are akin to a function call. In addition to this standardized conundrum, you have browsers discrepancies with the APIs available inside a worker varying between vendors [1].
On node, you have the cluster API, which allows managing several processes so it's even further from light threads. On the bright side, most (all?) APIs have both a sync and async version.
As a result, there's nothing you can use for multithreading in a module without coordinating with the whole project. I think JavaScript needs language support for light threads, not APIs for multiprocessing.
[1]: https://developer.mozilla.org/en-US/docs/Web/API/Worker/Func...
I personnally dont like generators for concurrency.It's the wrong solution to the problem.For now I stick with promises,and I hope JS gets async/await keywords quickly.
Agreed, but what I meant is that the API (running a file in a worker) is inconvenient, first because there is no closure. Go allows the go routine to read variables in scope and has a race detector to tell you when your accesses are unsafe. There's also the issue of managing workers: is a web worker an OS thread or a light thread [1]? how many can you spawn? which worker should you send work to? These problems are solved by Go. I think it would be very hard to obtain the same reliably and performance with today's web workers. Other languages solve this with a language construct, not an API to run another file.
> I hope JS gets async/await keywords quickly
It would be a step up. But most of the time you want to express a linear list of tasks. Javascript should support this by having a synchronous variant of its APIs and having light threads.
My hope is more that some language having such features will compile to asm.js
[1]: it's either a thread, a process or an "equivalent construct" http://dev.w3.org/html5/workers/#processing-model
If you consider that we have PNaCl compiling llvm-bitcode to native within the browser, Emscripten cross compiling llvm-bitcode to asm.js (which Firefox's engine converts to native), Safari's engine compiling js itself to llvm-bitcode, I'm thinking why not make llvm-bitcode an open web standard? That way, any language that compiles to llvm-bitcode would be equally usable on the server side and on the client side.
The biggest potential downside to that would probably be educational. I learn a lot by looking at source code and the web was always open in that way, though with minified JS these days that's not so possible any more. However, it might make it easier to distribute source code in any language, since a module can then compile and run such code right in the browser.
edit: These thoughts came up when I was thinking about how TJ won't be able to take his work with him as he moves to Go. Well, I suppose he wants to do new kind of work for which Go is better suited, but is then all of the prior work just a brief view along the journey?
...but I see that's a very specific comment:
the tooling Go provides for profiling and debugging is great...
As it stands, there's way too much friction and it would cost a LOT of money to invest in Go. There's a good chance that the ecosystem may not evolve to Node's level for years - and worst of all, if you want to build competitive web apps, you still can't avoid Javascript.
Hope the departure of an incredibly js developer doesn't impact the community too badly.
Hmm, do I pass a string in the first parameter, or was it the second? What does that funciton return? Am I really getting a number in this parameter? Of course, you can get around that confusion by adding more unit tests and parameter validation but with strong typing I'd get all that for free.
At my shop, we emphasize making obvious (if somewhat verbose at times) names for functions, so that it's pretty easy to intuit what arguments it takes. Also a naked function (e.g. "foo(bar)") will always be defined in the same (<100 line) file, so it's not hard to find it, or in a module (which is a Cmd+P away).
All this to say, while I agree with you that strong type systems mostly eliminate this pain, it's not exactly "painful" in Javascript either.
One tip: rubber ducking is really valuable. If it takes more than ~30 seconds to explain whats going on in ~10 lines of code, then you probably should do some abstracting.
The bigger reason I see for sticking with Node.js for web development is being able to use Javascript for both server and client code.
Newer Javascript frameworks and templating libraries such as dust.js, Ractive.js, React.js can render both on server side and client side so on the initial requst the server can send a fully rendered page for SEO and performance, and then client side Javascript can dynamically update the page using the same code/templates.
really means
"I am ready to trade speed of development for having more bugs"