Node.js is forked, not fucked | Hacker News Reader

Node.js is forked, not fucked

wesleyio.tumblr.com

138 pointsjosep268 comments

Open articleView on HN

This piece is (by far) the best written on the io.js fork and its ramifications. As I have said many times before, we are forkaphilic[1] -- forking is healthy because it allows (and encourages) experimentation with divergent ideas. The primary idea that io.js appears to be experimenting with is "more frequent releases", in particular very closely tracking V8. This is somewhat ironic because the close tracking of V8 has been a major contributing factor to the delay in 0.12: despite good intentions, V8 breaks node.js frequently -- and debugging those problems is a time-consuming and arduous task requiring a great deal of technical sophistication. We have made clear that our technical priorities for the project are production-grade stability and performance; to the degree that other have different priorities, forking should be encouraged. In part to allow cross-pollination from any fork, we eliminated our CLA earlier this year[2]; the community can rest assured that good ideas from either will be able to be in both.

[1] https://www.youtube.com/watch?v=Pm8P4oCIY3g#t=23m43s

[2] https://www.joyent.com/blog/broadening-node-js-contributions

> despite good intentions, V8 breaks node.js frequently -- and debugging those problems is a time-consuming and arduous task requiring a great deal of technical sophistication

Can you provide some examples? I'd love to see some debugging war stories.

TJ can provide examples from his perspective; my personal examples are all from keeping the node.js production debugging facilities[1][2] operating properly. We have tried hard to make these facilities independent from V8, but the reality is that they encode how V8 operates and that changes to V8 can break them. Fortunately, the node.js DTrace ustack helper[3] -- by far the more intricate piece -- hasn't needed major maintenance, but the facilities for iterating over all JavaScript objects from a core dump (for example) have been broken quite a few times. I'm afraid the war stories from debugging these aren't terribly heroic -- they boil down to one of the node.js debugging tests failing and Dave Pacheco or me (exactly which one frequently being adjudicated by fingers-on-noses) sitting down for several intense hours to learn how V8 has changed and modify our debugging support accordingly. If you want to see the details of the changes needed to accommodate V8, you can look at some of the specific commits for the debugging support.[4]

[1] http://www.joyent.com/developers/node/debug

[2] http://www.slideshare.net/bcantrill/goto2012

[3] http://dtrace.org/blogs/dap/2013/11/20/understanding-dtrace-...

[4] https://github.com/joyent/illumos-joyent/tree/master/usr/src...

Thank you, I will definitely check those out.

What is node core devs relationship with V8 team and Google,if there is any?

That's what makes me the most worried with nodejs.It depends on a project led by a corporation that is difficult "to read".We're not talking about a C++ lib that has been stable for 20+ years,but something that is likely to change fast and frequently,which will break every native extension written for node.

It seems to me that Goggle isnt that implicated in node development.

Thank you! I'm glad we were able to share a brain wave. I'm loving the term 'forkophobia', and the comparison to Berlin. Freedom to choose means that people will gravitate to the best ideas. At the end of the day, everyone wins with open source.

It seems to be the issue is a little more than "more frequent releases" but is instead more about the poor stewardship of Node.js by Joyent. It seems to be about increasing transparency and openness instead of just tracking v8 more closely.

Between io.js and node-forward it seems to be working. Hopefully things like the node advisory board are just one step in Joyent being a better steward.

Forking is healthy. In the late 90s/early 00s, you'd hear about a new Linux fork almost every other month. Those forks would die quickly, but sometimes would have something to contribute back to the larger distros.

The fact that Node is popular enough to fork is a good thing. There is an inherent disagreement between Joylent and the Io.js folks, not on technology but process, and it's likely that the conflict will be resolved that way they always are (the more suited fork will win out). The risk of fragmentation is very low.

There were some notable fork successes which overtook the original; egcs is the one that comes to mind. There's still a huge variety of distros, mostly RPM-based. And the three BSDs. And Lucid/Xemacs.

To clarify, Xemacs is a fork, but almost nobody uses Xemacs any more.

YUP! -- when I heard it got forked my first reaction was "great, good luck to them, I'll continue to use node, imagine moving to Go and maybe in a year I'll hear about iojs if it goes anywhere"

Cheers to the iojs guys.

(ps. I have to admit, I hope to see more blog posts on these subjects creatively include "motherforker" in their titles)

I think the linux comparison is quite apt. Even now, there are multiple distros that all use the same package format, deb or rpm. They are different forks of linux, but they've made a point to still have interoperable packages with one another.

The only thing I worry about is while packages will be npm compatible, if they want to take iojs to another level as far as features go you will then have npm packages that either only work on iojs or you will have polyfill libraries for iojs features to work on node.js.

Yeah this doesn't mean node.js is "fucked" but if they use npm in iojs with packages that take advantage of new API calls any node user is going to have lots of issues.

Edit: Having said that I am excited to possibly get newer versions of V8 server-side; I can't wait to start using some ES6 in a place where I don't have to wait for browsers to update.

If you can implement the new features with polyfills, why do you need io.js in the first place?

Better performance, debugging, and a simpler developer iteration loop for polyfills that require a compilation step.

Having them used out of the box would make them more robust.

NPM dependencies are a very robust method for ensuring that a module has what it needs to function "out of the box". What else are you looking for here?

Do you mean to reply to me? I made a commeny about innovation in the stdlib would be good. and I'm quite happy with io.js supporting npm.

>>>>If you can implement the new features with polyfills, why do you need io.js in the first place?
>>>Having them used out of the box would make them more robust.
>>NPM dependencies are a very robust method... What else are you looking for here?

>Do you mean to reply to me?

In light of nawitus's comment, I interpreted your comment as an alleged reason IO would be necessary, given the existence of polyfills. This reason didn't make sense to me, because npm dependency installation for node is just as automatic as anything in IO could be, whether it uses npm or not.

io.js is doing new things, they might be polyfillable on top of node, but more people will use new versions of inbuilts like 'process' of 'fs' if they're in io's stdlib than if they're in some optional module people could (but probably won't) install on node.

Performance comes to mind. Robustness too.

Those are decent points, but I'd think Io.js would have more reasons.

First of all, if you implement a polyfill in JavaScript, it'll be translated into pretty fast native code. But even if it didn't, most(?) of Node's core libraries are written in plain JavaScript and presumably are as fast as any module in npm. Take a look: https://github.com/joyent/node/blob/master/lib/buffer.js

And if you really want optimized native code, npm supports that out of the box. I kinda dislike npm modules that require a compilation step, but they're out there.

As for robustness, there's nothing stopping one to making robust node modules that are not in the core.

FWIW, https://github.com/petehunt/node-jsx will let you use many of the JS Harmony features (lambda arrows, destructuring, concise method notation) in Node today.

These are all the sugars that are available:

https://github.com/petehunt/node-jsx

That looks pretty nifty; I don't have an immediate need for it yet but I'll definitely check it out. Thanks!

There's already some of that... there are packages/tooling built aroud co/koa with features that you need the latest 0.11.x and --harmony to use.

I like a lot of the ES6 features... to ES7, if await/async can be introduced with thenables sooner than later, it will mean that co will become irrelevant for the most part, and a lot of tooling around a koa2 will be awesome. There's a lot to be excited about.

Forking is forking. Most forks are benign. Some forks may turn out good. Some may turn out bad. Forking is neither good or bad. Forking is forking.

It's easy to paint Joyent as the issue, but the root cause is the core contributors are split on their expectations and how they choose to handle them.

Sounds like a healthy split to me. Let Joyent handle the risk-averse companies that require a stable tool, and let the community explore the future of V8 server-side javascript. I'm sure some of iojs' successes will find their way into node.js as well.

> Let Joyent handle the risk-averse companies that require a stable tool.

Node is stable.Like too stable without even hitting v1.

I remember a video where Douglas Crockford predicted that,saying Joyent behavior has been childish.

http://youtu.be/8HzclYKz4yQ?t=22m32s

> “The last thing we want to have happen to the Node ecosystem is to create a Python 2, Python 3 situation”.

As a long time Python user unhappy with the 2 vs 3 thing, is not the Node vs IO thing very different? A fork (Node) vs two versions being managed by the same group (Python)?

At least for now, iojs works with node and node works with iojs. They're also managed by different people.

It's different but if IO/Node diverge too far, it could break packages.

They're both MIT and likely to stay MIT. They don't have to diverge at all unless Joylent goes open/closed.

Edit: Downvoters, have I said something that is untrue?

But isn't the point of the split that they have differing opinions on how to do things? Hence, they -will- diverge according to those differences in opinion. Not to mention, keeping them consistent means someone has to take code added to one, and port it into the other. That must take some work, and I can see the laziness of people coming through.

The whole debate is over uptake speed. If async functions and lambda arrows land in IO and I use them in my package, people on node v10 aren't going to be able to use it.

If Joyent accepts new features as quickly as IO, there's no point in the fork.

Essentially, there's bound to be some level of fragmentation.

I don't really see one implementation being a superset of another as fragmentation - IO will just be better, and Node will slowly die.

What this scenario means is that there should be a way for a module to depend on polyfill modules, only for particular engines. So on IO the dependency would not be in effect, while on e.g. node v0.10 it would. I don't think this is possible now, but you could just publish two modules.

That's already happening... co was released over a year ago, and still requires the dev version of node and the --harmony flag... this is something that should be in 0.12 which should have been released by now. Any modules relying on generators already require gnode or > 0.10 ... so uptake speed is very relevant and already an issue.

It's about how slow things have moved since 0.10 as much as anything else.

It'd be a very similar experience for users, where IO code/packages would be incompatible with node.js code/packages despite being, on their face, very similar.

But you're right, that situation might not last as long, due to the fact that one is theoretically trying to eat the other's users, rather than in the case of Python, where things are much friendlier.

With both libraries MIT licensed, there could easily be movement of code from io.js to node.js once the team at Joyent were satisfied that the code has been done correctly.

I fail to see how there can be controversy over forking MIT licensed software.

Very reminiscent of merb (a wycats led divergence from Rails from about 5 years ago). I expect a similar arc for io.js: a couple of distinct technical fetures that lead to a brief period of rising developer mindshare, then in a upcoming version node.js will promise to embrace/extend whatever makes io.js "better" and the project will peter out.

It's not technical features so much as an open governance.

More apropos is egcs, where the FSF eventually abandoned their gcc and blessed egcs (and their maintainers) as the official gcc.

This is the first time I read about this drama, can somebody explain to me why getting forked is bad thing..?

Possible community fragmentation until one fork "wins.

Well at a minimum you are dividing the development efforts between two different pieces of software. Which means (arguably) net contributions for either one should be less then if all contributions were going to a single project.

Forking is the sexual reproduction of the software world. Yes, just like the living world it takes a lot more resources to reproduce in this method, you end up with a much larger number of variants. These variants can fill various niches, such as security, speed, or ease of use. Most variants will get neglected or die off, but many will succeed, and some may be reincorporated into the original project (cross-breeding).

Wouldn't forking be asexual reproduction? Only one parent...

Yes, but with horizontal gene transfer.

But asexual reproduction can also be known as cloning! Just a bad/unfortunate analogy.

Yes, binary fission more specifically

It could also be self-fertilisation, which is still a form of sexual reproduction. Sorry for being so pedantic.

But in all likelihood, most forks will be worse along all axes because they have less development put into them.

You can't assume that development is going to go into the original fork when your core developer set is unhappy enough to create the second fork in the first place. If some hypothetical third party could mandate that no fork took place, you may very well find in your alternate universe that the development effort you are expressing fear may be split may simply go nowhere at all. You can not simply take it as a given that the developers are simply "stuck" with the original fork and if unable to do their own fork would simply contribute to the original.

No, I can't assume that — and I didn't. I didn't say anything remotely like that.

I was responding to a comment which said, essentially, that you will end up with lots of forks that are each strong in some particular way. This is not generally how it works in my experience. In general, the fork with (for example) better ease of use will also be the one with better speed because it will get much more time and attention than other forks.

I'm pretty sure you didn't read my comment at all, "Most will die off" was there for a reason. Also, don't confuse a fork dying off with it becoming something completely different.

I'm sorry if I gave that impression, but just because I didn't respond to those four particular words doesn't mean I didn't read your post at all.

I have to believe "These variants can fill various niches, such as security, speed, or ease of use" and "many will succeed" were also there for a reason, and those are what I was responding to. My point is that for most projects, you don't end up with a bunch of viable forks. It's rare even to end up with two viable public forks, much less several. It's not much like sexual reproduction because sexual reproduction requires there to be multiple breeding entities.

> Well at a minimum you are dividing the development efforts between two different pieces of software.

Well, no. Because the realistic alternative to forking isn't that the people with irreconcilable differences that led to the fork continue working on the same code base, its that some subset of them abandon the code base altogether. Forking -- especially when the forks are under compatible open source licenses and useful code that is compatible with the vision on both sides of the divide can migrate from one fork to the other -- potentially increases, rather than decreases, the effective development effort on both sides of the forks compared to what it would otherwise be.

The reason why it seems like a really bad thing in this case: many top contributors left node over some core principles.

But I agree with the article, when you read deeper into the comments and commits, it's not virulent in the record (though I'm sure tempers flared).

That being said, I'm just a lowly web dev and barely understand what's going on under the hood.

"The future of javascript looks bright, sunny, and full of callbacks. I’m excited." <-- Great opening

I prefer a different promise of the the future.

There's no way to work "coroutine" into a natural-sounding sentence, but that's Brendan Eich's vision of the future of async handling in JS.

I'm going to defer responding.

Just a quick q: does it involve a bluebird on your shoulder?

https://www.youtube.com/watch?v=HDZs0S7qd64

does anyone know where Ryan Dahl is these days? I'd be interested in what he has to say.

Link should be changed to permalink: http://wesleyio.tumblr.com/post/104637877991/node-js-is-fork...

I was expecting a screed against juvenile sexual innuendo, but this was good too.

One guy gets paid to write about meaningless micro trends, another guy gets paid to blast him. Nothing to gain from this in my humble opinion

Reply on news.ycombinator.com