The Future of Programming in Node.js
groups.google.com
groups.google.com
I'm sorry, I just can't get over it. "Just solve the problem" - I hate that attitude. That's how shitty software is born. Incompetent schmucks give birth to unholy, unmaintainable, monolithic code because they think it's okay to not know your shit as long as you "solve" the problem.
Note also that Ryan is a mathematician. Mathematicians hate engineering because there's nothing to protect real-world systems from little accidents that compound over time. No proof, no piece of code, no matter how elegant, can reverse the thousands of bad design decisions that have resulted in the modern Linux ecosystem.
---
"The only interesting languages are C/C++, Go, Dart, JS (and possibly Rust). Everything else is legacy bullshit."
"Node.js has linear speedup over multiple cores for web servers.”
There is also the one about math not being useful for computer programmers and how programming is just like playing with duplo block and legos.
---
I don't know. Node scaling linearly over multiple cores. I can see how many would agree with you saying he created node.js but might think twice about the schmuck label.
Look, you can't just throw cheap shots, that's all deuche. What was the need for calling him an idiot or incompetent?
But, I would not want Ryan writing code that people completely trust to live on (hospital equipment, NASA, etc).
That does not make him an incompetent person. Just someone with opinions of wanting to keep things as minimalistic as possible which is just one of the many paths available for us to choose in life. In some cases, this is great, in others it is not.
Just like I wouldn't expect a civil engineer to build a bridge in a suburb capable of holding 5 Tons because "NASA does it that way because their spaceships are heavy" I wouldn't expect the same guarantees from software engineers writing an image sharing application.
Diversity is good anyways.
Hell no. Software comes from software development. And software development is a process. My editor, programming style, directory structures and other personal organizational behaviours do matter. This is work we do, and while the end user's experience may be identical regardless of very different personal development style options, the one which gives me the most comfort and work satisfaction is superior. So yes, I will configure my editor so that I don't have to keep repeating the same keystrokes dozens of times throughout the day. I will align things the way I see fit, and I will endeavour to study and improve my skill with the language. Not only so that I'm more efficient, but because it will bring increased contentment to the process of development.
Only nobody cares about you.
As in a restaurant, people care about the food, not the cook.
And even if you (and I) mattered, developers of a piece of software are N, where users are 1000N or 100000000N. They matter massively more.
Oh, and there will always be programmers as long as there are money to be made, so the "without me [in particular] there would be no software" argument doesn't hold much either.
The health inspectors care about both the food and the cook, and the latter's workflow is certainly scrutinized.
If we're going to beat this analogy to a bloody pulp, the health inspector in software land would be technical debt and colleagues.
Not taking care of the process will bury the project, and the users with it. Ryan Dahl's brilliant, but I don't think his entire statement should be adhered to verbatim. It's solid guiding principle, nothing more.
What a different industry it would be if all software required independent audits and the public could get access to the results.
I care about how code is written, because I'll have to work on it. My employers care about how code is written, because our reputation is on the line.
It's about solving problems, certainly, but workflows directly contribute to solving problems better, faster and more reliably.
Similarly companies too crazy about short term shareholder value find that it takes committed employees to get long term results.
Except me, I care about me.
And I don't want to use crappy and shitty tools. I want to use good tools. I am less productive when I use crappy tools, it can be done, but then a lot of time is spent fighting the tools not really solving problem.
> Oh, and there will always be programmers as long as there are money to be made
There are not that many good programmers. Or at least they are not the ones throwing resumes around. One has to find them, if they are good they are probably already in a good place and are not looking for job. So you just taking anyone from the street because they did some VBScript in High School you are going to replace one good programmers with even 100 incompetent ones. It is just not how it works.
Neither does Ryan. That's what the whole rant is about. He just wants to make software for users without having to hack through the forest of bad design decisions that is the modern Linux system.
Spoiler: Most software you create wont be around in 10 years: http://www.youtube.com/watch?v=zut2NLMVL_k&feature=youtu.be
What matters is the value it delivers in the meantime, which is ultimately the end-user result - not how you got there.
Yes, in the end, the software won't be around, in fact nor will those who derived value from it. All that will matter is how you got there.
>Spoiler: Most software you create wont be around in 10 years
I was working for a start up about about 5 years ago working on a the reporting system for measuring customer metrics, the data was granular enough that we allowed the customer to drill down to the per-hour activity level. As a result it was deemed important to represent the data in the customer's timezone. To "future proof" the system we had a table indicating when to shift an hour for daylight savings time. I populated the table to go out 25 years I think. My manager asked me "what happens in 2033?". But after a moment of thought he came to the same conclusion you did and 25 years was deemed sufficient.
That is true, but the experience of the people developing the software matters too, as it informs the care that will be put into creating a better user experience.
Let's say you're creating an application and you're the sole developer and designer of that application. When you have a predictable and reliable development environment that emphasizes good, well defined practices, you reach a state of flow with more ease when you're coding, and your project gets developed faster. Faster development allows you to iterate quickly and better test your assumptions about the user interface and the overall user experience. It gives you more time to design the application. All in all, with a better development experience, the developer gets more time to create a better experience for the user.
I bet Ryan's frustrations—all the abstractions and crappy interfaces—come from years of legacy technologies and the need for compatibility and integration with other systems, and sometimes as preemptive measures to avoid security issues. When they were made, there was a reason for their existence.
http://www.vpri.org/pdf/tr2010004_steps10.pdf
STEPS Toward Expressive Programming Systems, 2010 Progress Report Submitted to the National Science Foundation (NSF) October 2010 Alan Kay et al
"If computing is important—for daily life, learning, business, national defense, jobs, and more — then qualitatively advancing computing is extremely important. For example, many software systems today are made from millions to hundreds of millions of lines of program code that is too large, complex and fragile to be improved, fixed, or integrated. (One hundred million lines of code at 50 lines per page is 5000 books of 400 pages each! This is beyond human scale.) What if this could be made literally 1000 times smaller — or more? And made more powerful, clear, simple, and robust? This would bring one of the most important technologies of our time from a state that is almost out of human reach—and dangerously close to being out of control—back into human scale.
(...)
Previous STEPS Results
The first three years were devoted to making much smaller, simpler, and more readable versions of many of the prime parts of personal computing, including: graphics and sound, viewing/windowing, UIs, text, composition, cells, TCP/IP, etc. These have turned out well (they are chronicled in previous NSF reports and in our papers and memos). For example, essentially all of standard personal computing graphics can be created from scratch in the Nile language in a little more than 300 lines of code. Nile itself can be made in a little over 100 lines of code in the OMeta metalanguage, and optimized to run acceptably in real‐time (also in OMeta) in another 700 lines. OMeta can be made in itself and optimized in about 100 lines of code."
More:
http://www.vpri.org/pdf/tr2011004_steps11.pdf
"This year we are able to write this entire report in the Frank part of STEPS, including producing the PDF version for NSF’s web site. We can give comprehensive high‐quality, high‐resolution presentations using STEPS which include live code and in situ demos, and which show working versions of a number of the document types derived from the STEPS “universal document”."
So yes, they have something that is working. Unfortunately, as far as I can figure out - the "Frank part of STEPS" isn't really available as such.
There's a few thing to play with, like:
http://piumarta.com/software/cola/
http://lukego.livejournal.com/16036.html key quote: "Here's what a hello-world-esque operating system looks like"
There's also:
svn co http://piumarta.com/svn2/idst/trunk idst
head -2 idst/function/examples/tcp/00_README
This example demonstrates an alternate 'shell' for
Jolt and how to build a tiny TCP/IP stack using it.
Sure, down in the trenches it makes you want to pull your hear out. And the world would probably be more productive if software interfaces and programming languages were completely frictionless. But it would also make programming, to me, less fun, challenging, and enjoyable.
We are imperfect beings living in an imperfect world, and I don't think changing the status of either or the former or the latter would create the nirvana ry is alluding to.
The problem is that the wrong things are fucked and that's neither challenging nor fun.
E.g configuring some system so you can get your software built correctly is neither fun not challenging. It's merely tedious busy-work. Actually solving problems, that's the fun part.
One man's tedium is another man's pleasure, as much as I too enjoy solving the bigger-picture problems. There's a reason both Gentoo and Arch exist :)
And what you're saying is precisely what I'm disagreeing with. I'm sure there are people that like those things (mostly beginner's with Unix and such, tinkerers etc). It's a form of passtime.
That doesn't mean it's challenging in any creative way. It's a boring ass brain puzzle that tons of people are solving again and again.
The sense of "accomplishment" of getting the dependencies to build X right, might be valid if you just started building things and getting to know how compilers, libs etc work, but it's grows old very quickly for normal people and it's a false accomplishment after that initial stage.
Actually saying you enjoy that is like saying you like boilerplate code, because that's exactly what it amounts to.
>It might not be the kind of fun and challenging you're looking for, but please, try not to state state opinion as fact.
Sure, you can find people that like everything, even the most boring ass shit. I'm sure there are people that enjoy calculating logarithms manually too. But I wouldn't call such bizarro outlier fun as fun in the accepted sense.
But it's a fact (I've seldom seen anyone state the opposite, tons of texts from Stack Overflow to blog posts support this) that most programmers/hackers tear their hair out to get rid of those "fun" tedious configuration/setup/figuring dependencies etc work and move to the real part.
I've actually never read an interview with a hero programmer where he said that he likes those parts of programming -- but have read tons of those that say the exact opposite, from Alan Kay to Jamie Zawinsky.
Actually a large part of programming is exactly about removing (automating) those tedious parts.
By the way, your post comes off as unnecessarily rude.
> program runs today, we're doing everything we can to make sure that it
> will run next year, albeit faster and more reliably.
I've yet to devote any significant amount of time to developing on Node but this is a big +1 for more serious consideration.
My memory is fuzzy (this was well over a year ago) but I remember building an app in express and it being completely broken by updates a very short time later (along with a good number of other modules that hadn't been updated either).
Anyone venture out a guess/opinion on how many things in userland will stabilize because of this? (Or perhaps things already have- I don't hear as many complaints about express' api changing lately).
The community as a whole is pretty strict in it's respect for versioning conventions.
* module A depends on module C v1.0.2
* module B depends on module C v1.4.3
In all the languages you mentioned it becomes a pain because you can only use one version of module C, meaning either module A or B simply will not work until you find a way around it.
Semantic versioning's raison d^etre is to prevent these sorts of issues. Reusing a major version number is supposed to constitute a promise not to remove or change the call signatures of any functions published by your library. This is necessary so that a dependent library can declare a dependency on version 1.x.x of your library when version 1.1.0 is released, without having to worry that version 1.2.0 of your library will break things.
The problem is that too many library and module authors (who are otherwise talented, or are simply the first provider of a useful library that ends up gaining traction) refuse to follow the rules, and there's no effective sanction in the OSS marketplace for this sort of antisocial behavior.
As soon as backward-incompatible change is introduced without bumping the major version number, the dependent module author becomes paranoid (and being closer to the user, he's going to wrongly get a disproportionate amount of blame), and (rightly) feels he has no other option but to declare a strict version dependency. And when there is more than one dependent module involved, the misbehavior of the independent module author can cause a dependency graph that is impossible to satisfy.
As far as I can tell, this whole situation started with Ruby, and is the main reason (along with second-class documentation) why I am generally averse to its ecosystem. rvm and its ilk shouldn't even have to exist.
Semver may surface them by making it very clear (assuming all involved libraries use semver) where they can occur, but, if you have a package management/loading system that only allows one version of a particular package to be loaded, obviously can't do anything to prevent the situation where different dependencies rely on incompatible versions of the same underlying library.
Sure, with semver it won't happen if A depends on C v.1.0.1 and B depends on C v.1.4.3 (as A and B can both use C v.1.4.3), but it will still happen if A depends on C v.1.0.1 and B depends on C v.2.0.0.)
To actually avoid the problem, you need to isolate dependencies so that they aren't included globally but only into the package, namespace, source file, or other scope where they are required.
As for "rvm and its ilk shouldn't even have to exist": you do realize that rvm and its ilk are not just to allow you to pin your software to a specific Ruby version, right? They're also there to allow you to easily upgrade to newer versions. Let's face it, compiling stuff by hand sucks, and your hair has turned white by the time the distro has caught up.
This is not strictly true in Java. You can set up your own classloaders that allow you to load multiple different versions of the same class and hand out the right instances on demand. (This requires some work of our own since by themselves classes are not versioned. But you can evolve simple versioning schemes on (say) Jar files to solve this)
Obviously not trivial, especially if you are rolling one on your own. But you can use an OSGI implementation to do most of this for you in a standard way. JSR 277, if and when it is implemented, should provide another solution in "standard" Java.
Not really sure about python and ruby.
Things are even more twisted when you have a half dozen versions of C floating around in your node_modules, and the problem isn't in your code, but a dependency of a dependency.
Another issue I've run into is patching a bug in a module, and then having to figure out how to get that patch into all of the other versions that cropped up in node_modules.
NPM is one way to solve the modules problem, but it's no panacea.
So, it solves some headaches, and creates others.
I feel like your complaints are a user problem. I don't have the "too many files" issue when I use vim.
Also, gitignore does not work in SVN (*omg he uses SVN! the shame!), and the node_modules do actually have to be included in the source since the runtime is disconnected from the internet (intranet app).
Like I said, I didn't spend much time with them. They are a solution. NPM feels like the old rails gems and is certainly better than most package managers. Requiring external libraries and files specifically is a hack and feels as such. I would gladly take any of the languages inclusion techniques I listed above over node's.
Edit: I am not trying to put down the hard work that went into creating the system. JavaScript does not support modules natively, so adding support is a momentous accomplishment. The unfortunate reality is that it will always be a limited solution. JavaScript modules (in any form) are going to be inferior to languages that natively support such a basic part of programming.
Coffee script being a hack built into a new beast of a language, I have yet to read docs that drift from "JS is better! ... No, CS is better!"
Use the right tools for the job, and don't complain that your pentagonal screwdriver doesn't always work on phillips head screws.
IIFEs are also pretty orthogonal to Node's module system — they work perfectly fine together, and don't have much to do with each other at all. Not sure why you'd feel the need to disable IIFEs in the CoffeeScript compiler except out of desire for cleanliness since they're an ugly hack in the first place.
Personally, with JavaScript being the first language I seriously learned, it's intuitive for me to reason about. JavaScript is built around closures, and you're simply importing a closure that returns (exports) its public interface.
In fact, the main issue I have with ES6 modules is that they differentiate a "module object" from its default exports, which can be tricky to reason about (basically, what I wanted from ES6 modules was just syntactic sugar over CommonJS, whereas what we have is more powerful but more complex).
I have nothing but praise for NPM - it's simple, fast, keeps dependencies isolated (but can dedupe if wanted), and did I mention fast? It runs circles around any other package manager I've used, except maybe for homebrew.
A project file has references [1], which define the project's external dependences, Removing references is easy. [2][3]
[1] http://msdn.microsoft.com/en-us/library/ez524kew(v=vs.110).a...
[2] http://msdn.microsoft.com/en-us/library/hh708954.aspx
[3] http://msdn.microsoft.com/en-us/library/7314433t(v=vs.90).as...
But most importantly as Barbara Liskov mentions in this video[1] we don't know what is a module exactly or how to use them yet. Which is an specific statement aligned with Alan Kay's famous "We don't know how to design systems so let's not turn it into a religion yet."[2]
tl;dr; 1) Innovation is good. 2) Javascript's module is a half assed implementation of Common Lisp's defpackage. (Don't get me wrong is still way better than Python's abhorrent ninja goto: import)
[1]: http://www.infoq.com/presentations/programming-abstraction-l... [2]: https://www.youtube.com/watch?v=oKg1hTOQXoY
Computing will need to move fluently between the cloud and the terminal, and JavaScript is the only language which allows you to use a single codebase for both (in the foreseeable future). As developer time will stay the biggest expense when developing software, I am bullish on Node and JavaScript.
Because Node does not strictly equal Javascript-- V8 is not Javascript, nor is spider monkey. Most node applications, without some modification, do not run in the browser. They are missing tons of libraries and cannot interact with the file system.
I do appreciate the ability to have a single codebase, but over the past few years, I've pivoted and hope to someone's god that it is NOT javascript. Mostly because, in my experience, Javascript itself, even with a great focus from the beginning, is incredibly difficult to maintain.
Even then, is having a codebase in one language the best idea? There are some things that are much better to write in Erlang than they are C, and vice versa. It depends on the problem you're trying to solve.
The bigger the project, the less Javascript I want in it.
I think ASM.js, or something similar, will be the death of writing Javascript. Then it won't matter what language it's written in, we can use source maps to debug and it'll be faster than we could've ever written by hand*
*single one off functions don't count. Try writing a webapp in pure x-86 assembly.
[1]: ocsigen.org/js_of_ocaml/
The argument for developer time--as well as decreased maintenance cost--is exactly why I went with OCaml over pure JavaScript. And so far it's certainly paid off!
Besides, the ability to compile to JS is not enough; you will need to read it in JS, debug it in JS and let other developers work on it in JS (because they don't know OCaml).
This seems short-sighted, with ES6 modules now being used in production (through transpilers) and slowly making its way into SpiderMonkey and, more importantly, V8.
https://gist.github.com/domenic/4748675
https://gist.github.com/wycats/51c96e3adcdb3a68cbc3#importin...
(both of those are slightly out of date, syntax-wise, but are still conceptually current, IIRC)
Thank the jesus. This will be nice to have, especially for some command line and filesystem operations.
> asynchrony. Generators and Promises are interesting and
> will remain a userland option. "
I am seeking for a comparable alternative to node (or something on top of node) in which Promises are the de facto way to implement async. Any suggestions?
You can totally write your node applications using only Promises using something like Q if you feel like it.
On top of that you get to use a much nicer language than JavaScript.
http://swannodette.github.io/2013/07/12/communicating-sequen...
Sigh... Let's leave this "future of programming" to the future then. I hope someone needs it.
Give it a snazzy name. Maybe.. "Livewire".
I don't think it would be very hard to to rewrite livewire as node module though, if you were really interested in it.
Is it similar to Go or Angular @ Google?
One thing I love about Rails is that it exists quite separately from 37signals or any other company. I don't get that feeling with Node/Joyent, but maybe I'm wrong.
The active core committers over the last month, and their employers, in order of whether or not they're me:
- Isaac Z. Schlueter (npm author/admin, lead gatekeeper, stream maintainer, javascripter), Joyent
- Ben Noordhuis (issues list champion, most opinionated C++ developer, libuv unix guy), StrongLoop
- TJ Fontaine (build owner, C shim author, tracing afficianado, Master of Robots), Joyent
- Trevor Norris (performance perfectionist, master of dirty hacks), Mozilla
- Fedor Indutny (russian ninja rockstar, TLS tamer), Voxer
Also very important to the project are Bert Belder (Windows guy, libuv co-author) at StrongLoop, and Scott Blomquist (Windows bug-chaser, Microsoft liason) at Microsoft.
Honorable Mention to Ryan Dahl (Node.js creator, BDFL) who is currently off being a professional hipster somewhere in NYC, and not involved with the project. When he did work on Node, he was paid by Joyent, and he got some money for selling the copyright to the company as well.
The Node.js community experience would be much less pleasant if not for Nodejitsu's continued support of the public npm registry. (Nodejitsu acquired IrisCouch a few months ago.)
Node would continue on even if Joyent bit the dust. (Depending on who assumed ownership of the mark, it may or may not have to change name/logo.)
If you've had a bad experience in the past with Joyent, it probably has nothing to do with the experience you might have with Node.js. I am given free reign here to manage the project free of any overt Joyent influence. Joyent is a big Node user, and a lot of really smart people, so I do take it seriously if they have something to say. Especially considering that most of the time when Joyent engineers have a problem with Node, it's related to other users hosted on their platform (Walmart, Voxer, etc.)
Joyent has basically got it as far as they needed it to get for their own uses. Now they are just sitting on it and keeping it stable while letting userland modules cover future expansion.
It would be cool if someone makde a Node.C++. I would have less of a learning curve.
anyway.. looking at the source its pretty decent, and go right to the point.. of course its focused in HTTP/REST..
theres a bultin IO scheduler and a concurrent task system.. from microsoft.. who would say so :)
I agree 100%. But still, a lot of people are complaining about some of the Node.js API choices.
They are complaining because it makes programming more difficult , and the truth is , async programming is hard when it relies on callbacks.
I've never seen anyone argue that callbacks are one of the easier ways of doing async.
So yes, grandparent is correct.
I feel like promises don't need to be part of core though. It's easy enough to wrap the standard library in userland and just have other things depend on the wrappers.
Awesome.