A Node.js hacker asks what Haskell can do to stay relevant. | Hacker News Reader

A Node.js hacker asks what Haskell can do to stay relevant.

stackoverflow.com

90 pointsKirinDave59 comments

Uh... it's very much the other way around. Haskell's building the future of concurrency, even if it isn't going to be "the language" of 2020, and Node.js is a throwback to late-1990s to early-2000s solutions that repeated and consistently failed to work at scale, with no real chance that Node.js is going to escape the same fate any more than the several dozen event-based frameworks based on languages very similar to Javascript before it did. (If the manually-compiled event callbacks, cooperative-multithreading scheduling approach was going to be the Next Big Thing, it would have happened years ago.) It's Node.js that needs to worry about how it's going to "stay relevant" as it walks a dead-end path, not Haskell as it blazes (relatively) new trails.

Do you have any examples of these late 90s early 00s solutions that consistently failed to work at scale? Can you point me to any of the 'several dozen' event-based frameworks before JavaScript did it? I don't seem to remember these very well, and if Node.js is just a throwback, I'd love to see what it was a throwback of.

Also, do you have any examples of these trails Haskell is blazing?

Twisted Python and Ruby's EventMachine would both be examples of evented IO frameworks that predate Node.js's popularity, and possibly even its development.

I think even C++ had one.

C++ has asio, now part of boost.

i have a bit of experience with twisted and must say it's pretty world class. it has great streaming APIs, really convenient APIs for working with processes, in addition to a solid networking library where you can either build your own protocols or use existing ones.

to say "it failed at scale" is ridiculous. i can point to a number of institutions with plenty of twisted code in production, moving hundreds of GB a day of data.

the only reason it would seem "failed" or "obscure" is that node.js has a huge fad aspect to it where people blog about and evangelize it constantly. most things in software don't have a fraction of the hype or exposure but that's not really due to technical shortcomings or lack of use.

"Failed to scale" means that async event-based programming becomes harder too quickly as the program grows in size. Performance isn't the problem, the problem is the program complexity explodes too quickly.

Twisted is world class; the fact that it essentially failed to escape its niche is one of the bits of evidence I submit for saying this style doesn't have some immense reservoir of success left to tap. Twisted basically tapped it, in a nicer language for what it is doing that Javascript is, and it's not enough to take the world by storm, in terms of solving the concurrency problem. Use it if it's the best solution, by all means, though. It's quality stuff.

i don't think anyone thinks it's a solution to the general problem of concurrency. it's an evented framework, much like django is an MVC web framework. use of either library is pretty pedestrian.

twisted made a few innovations (Deferred, @inlineCallbacks, push/pull producer API) but all in all select loops are not new technology.

node is getting way too much credit if you ask me, but i think it's mostly because it's exposed a generally naive class of programmers (javascript coders) to a different paradigm. C programmers have known about select() for the last 30 years.

Nested data parallelism:

http://www.youtube.com/watch?v=NWSZ4c9yqW8

Perl has so many that a library was built on top to manage abstracting between them, AnyEvent, which unified support for many event-loop based systems, most notably Glib, Event::Lib, and POE, but quite a few others (and IIRC POE actually outcompeted some other contender that is now dead). Ruby has EventMachine (predates Node.js, not that timeframe though). Tcl was simply built around an event loop, so it isn't even meaningful to talk about which library adds it. Python has Twisted and Stackless Python (which has many of the interesting capabilities that Erlang has at a language level), along with libevent and libev bindings. C also has numerous ones, with corresponding bindings in many languages, glib being one big one. gtk is built on glib, but you can get the glib event loop outside of gtk, and there are real things that use it, like the pidgin core, which is not actually dependent on a UI, only glib.

Actually, everybody has libevent and libev bindings. Everybody can get at event-based programming in whatever language they like.

Those are just the highlights. Also, if you're tempted to say "but what about this one library", go check to see if you can find it for Twisted first.

Also, it's worth pointing out that GUIs have been "asynchronous event-based programming" for over 30 years now, and in practice it's hardly any different in programming style than working with Node.js, especially when using a binding to Python, Perl, Tcl, Ruby, etc.

Haskell has one of the few functioning Software Transactional Memorys, and the reason why it works is actually critically based on the nature of Haskell. (Imperative languages have generally been unable to implement them because it turns out to be critical to control effects, and it's just too easy for something to slip in without a type system preventing it.) Haskell has "par": http://www.haskell.org/ghc/docs/7.0.3/html/users_guide/lang-... , section 7.18.4, which is nearly trivial in a pure functional language and a freakishly hairy mess in an imperative language. See also http://hackage.haskell.org/packages/archive/parallel/3.1.0.1... . Data Parallel Haskell is an interesting project in the early stages.

Actually, just watch this video: http://skillsmatter.com/podcast/scala/talk-by-haskell-expert... , which will also (at the end) explain what's interesting about Data Parallel Haskell beyond just "parallel arrays", which has been done; DPH is actually more interesting than that, and also something hard to imagine being implemented in something like Javascript.

Yeah, but.... i mean, it's Perl. EventMachine's HTTP server threw tons of warnings for every request when I used it in early 2010 (and that's actually what drove me to Node in the first place.) Twisted Python is pretty un-pythonic and building a huge webapp in C is usually a mistake.

To me, Node seems like the first (most?) "natural" way of programming in this style, even if the competition pioneered it. Everybody knows Javascript and lots of people haven't taken a look at your other choices, so it seems to nicely fit that gap in the programming language spectrum.

Thanks for showing off the haskell stuff. I really need to get around to learning that language.

Not having much experience with node.js, what makes it more natural than twisted ? I agree on twisted being unpythonic, but most of what I don't like in twisted seems rather fundamental to (explicit) async programming (callbacks everywhere, lack of meaningful stack traces, error handling extremely tedious).

"Yeah, but.... i mean, it's Perl."

If you get to play the "yeah, but", believe me, I can "Yeah, but it's Javascript" with just as much justification.

Given the choice between Perl and Javascript as it stands in V8, I'll take Perl in a split second. js.next would be a harder choice, but it's not the one I have.

"Twisted Python is pretty un-pythonic"

So what? Node isn't Pythonic either; is that stopping you? It's a great little talking point, but if you try to unpack it into something sensible there's nothing actually there.

The difference is Node has an ecosystem of async 3rd party libs for IO drivers etc. Your evented web server is worthless if your IO (DB, File, REST, ...) is sync.

Just as the difference between Java with the JDK and C(++) without any useful standard lib for application development.

"The difference is Node has an ecosystem of async 3rd party libs for IO drivers etc."

This is also an utterly false and misleading argument. You think these multiple event-based ecosystems were all staffed by drooling idiots who didn't realize that? This isn't some sort of subtle point that can be used as a base of a secret sauce, it's a blindingly-obvious core requirement. They all have drivers for all that sort of thing. Some more than others, but they all have the basics. Many were better stocked with more and higher-quality libraries four years ago than Node.js has now.

This is why I say I hate the Node hype, even as I'm only really ambivalent about Node itself. It's bullshit. It's like Ryan Dahl went into hibernation in 1995, only to pop out again a couple of years ago. It's full of lies about the state of the programming world. Node looks fucking awesome if you uncritically swallow the propaganda packet. Well, gosh, no wonder, it's like a magical 2010-ish library magically showed up in 1995! Node looks "meh" at best, and as I said, heading down a dead-end trail, if you look at where it sits in the real programming landscape in 2011.

"[...] but they all have the basics."

No. Perl had no async DBC drivers when I wrote Perl code. Netty does not have async JDBC drivers. But then what do I know. But then what do you know, your reply is essentially fact free.

It would help if you point me, e.g. to the 3rd party async IO drivers (JDBC, Redis, MongoDB, File - I have one for HTTP) for Netty.

Pretty much any GUI framework ever built, NIO, AIO, .NET eventing framework. All node.js brings to the table is a really verbose way of writing a lambda, it just happens to be a little easier in javascript than other C-derived languages. Want to see non-blocking IO at scale written in C? Go look inside any OS kernel.

Non-blocking IO is not a new concept. Haskell isn't blazing any particularly new trails. It's a refinement of ideas from functional programming, just like javascript is a refinement of ideas from imperative programming. Also, it's not really javascript that made Node.js possible, it's improved javascript execution performance.

It's just a language. Use whatever solves the problem you have in the way you want it solved.

I can at least find about 5 event-based frameworks focused on network in python alone without thinking much: medusa is ~ 15 years old, Twisted is > 10 years old, there is tornado (~ 2 years old), event/gevent (not sure about the history here), stackless (not sure about the history, but more than 5 years old).

Unless you are doing long polling, I am not sure I understand the point of using an evented framework (and even then, it could be debated I guess). It is very hard to code a significant and reliable piece of code with it.

> It's Node.js that needs to worry about how it's going to "stay relevant" as it walks a dead-end path

Yes, I believe that was the implied punch-line of the submission. http://mobile.twitter.com/kirindave/status/83252128575004672

Fair.

It's in JavaScript. If Haskell can replace JS on the client-side, maybe it could beat Node on that advantage.

There has been work done to compile haskell into javascript so this might be a possibility in the future.

http://lambda-the-ultimate.org/node/1836

I am sure there is also a project that uses a haskell->llvm->javascript but I can't find the link at the moment.

Haskell isn't building anything. It is just a programming language. It is people that are actually building these things and some languages, such as Haskell, are implementations of them.

I'm a big Haskell fan, but lambda-loving academia needs to get off its ass and spent a bit more braincycles on fulfilling Haskell's potential as a web language. Node.js community is simply far more energetic and ballsy. I'm quite confident both Node.js and Haskell are here to stay.

Why does academia need to do that?

There's a lot more to programming than building websites. Haskell seems a lot more interesting for solving problems that are actually hard.

I heard this argument a lot of times, both in defense of Haskell and Lisp.

Does Haskel ability to solve "hard" problems makes it somehow unfit for solving "easy" problems?

Or, to put it the other way, why do you think it can solve "hard" problems if it can't even solve "easy" problems?

Oh, I think it is perfectly possible to do webprogramming in Haskell. In fact, Haskell would be my first choice if doing web development. In my opinion the reason there is not much focus on web programming in Haskell is that (based on my observations in #haskell) the average Haskell programmer would rather work on these "hard" problems then on web programming.

Not to get into the whole Blub language argument, but I daresay the average Haskell programmer is more advanced then your average web programmer. This is not to imply the latter is somehow easier or trivial (working IE support? I'm not ever gonna call that easy....) but it could explain a different focus of the community. Resulting in less webprogramming being done. That said, I've seen 5i7 Haskell webprogramming frameworks pop up in the past 2 years, so I'm not the only one who thinks it is perfectly suited for that.

I wouldn't say average haskell programmers just want to tackle hard problems, I'd say it's more that average Haskellers are experts or have interests in very specific fields that don't tie in with the 'mainstream' programming mindset.

You give web programmers Rails instead of Haskell for the same reasons you give Jim in accounting Excel and Access instead of Rails.

Warp and Snap. Google them.

it's definitely being worked on

http://www.yesodweb.com/ http://snapframework.com/ http://happstack.com/index.html http://blog.on-a-horse.org/

I am sure there are more.

Yes, although some of those are more "worked on" than others; the last entry at blog.on-a-horse.org is from almost a year ago, and the page at haskell.on-a-horse.org has been broken for a while.

Saying "some" is somewhat disingenuous as haskell on a horse is the only one not being actively developed at this time.

Just to be clear, by "failed to work at scale" you are specifically referring to the asynchronous I/O codebases being difficult to manage, correct?

This is funny. I'm the person having submitted the question on StackOverflow. KirinDave just assumed I was a Node.js hacker (which is absolutely not true!). I really wanted to understand how Haskell is planning to attack the >10k-100k connections problem. The question was just an excuse to talk about the Haskell-way.

Personally, I think Node.js offers a regressive programming model and I'm concerned about too many peoples diving into it. Ryan Dahl is a very smart guy, doing a nice job and is greatly responsible for the success of Node.js. The rumors goes he would have been tempted to do something with Haskell first but he stepped back as he was not familiar enough with GHC. Actually, that not a rumor, that's what he said: http://bostinnovation.com/2011/01/31/node-js-interview-4-que...

The problem with Haskell is that Haskellers are very smart peoples (...smart peoples again!). They became very comfortable with some hard to grasp concepts for programmers having a more conventional pedigree and might underestimate the education effort required in order to attract more programmers. Most of them are not necessarily focused at creating a more accessible language but rather explore new FP concepts (which is great, BTW). Things need to change in university classes first.

So you have potentially hard to maintain callback code on the node.js side and monad/lazyness/FR to master on the Haskell side. Nevertheless, from a skill-improvement point of view, I think the Haskell option is a much better investment.

Back to my Scala book now... :)

I did just assume, and I apologize if that is offensive to you. But you have to admit, this title is pretty provocative. ;)

But the SO post says nothing about anything "staying relevant." It's simply, "What is the Haskell response to Node.js?" with some follow-up.

Dave, I had seen the Stackoverflow post before so I was surprised when I clicked the link here and got there, because your title misrepresents the question.

I thank you for posting it here because it has indeed led to interesting discussion. But please consider not editorializing titles in a way that can lead to flame wars.

I didn't think I was editorializing, and I apologized after it became clear I had misinterpreted the author. I knew the concept was controversial and interesting, so I posted it (and it was both interesting and controversial).

I'm afraid AccurateFuturePrediction.hs doesn't compile on OSX due to the size limits, so my ability to avoid this sort of gaffe in the future is limited.

No offense, it's just funny ;) Provocative titles are good, they create a debate. It's true that I was playing the devil's advocate.

Long title would be "Hey you Haskell guys, everybody is thrilled about node.js, why don't you explain how to achieve high scalability in Haskell without sacrificing the program structure. C'mon guys, I know you can!"

If Haskell is leading the way, they should have some response about node.js, right? Whether or not this can be mainstream and deployed everywhere is another question. I totally respect Haskeller position regarding the experimental/research status of the language.

Haskell is a great vehicle for cutting edge language research. What's the rush to push it into production coding? Why not let the better ideas trickle down into more pragmatic languages (Scala) and leave the Haskell implementors free to experiment?

Answer: if you know Haskell you don't have to run scared from threading, because threading really isn't so hard.

I've heard Node advocates argue that being limited to a single process isn't a problem for scalability because you can just spin up more Node instances... but that's just threading on a different level. You still run into issues coordinating the instances, but now they are going on in your database instead of in your code.

If you're eventually going to hit the upper limit of a single machine, have you really gained that much? Eventually you'll have to figure out a way to coordinate between processes, because some will be running on other nodes.

I think distributed computing is not a language problem. If you solve the problem on one computer its allready a big help if you build a distributed system.

http://qconlondon.com/dl/qcon-london-2011/slides/GregoryColl... is about this exactly. He explains why Haskell's green threads are better than Node's callback driven model.

The SO question and many of these comments are ridiculous because Haskell has already solved this issue in a much better way. Here is the benchmark: http://www.yesodweb.com/blog/2011/03/preliminary-warp-cross-...

Give Haskell 4 cores and it can do 100k (simple) requests per second in a single application. Node can't do as many, and can't scale a single application across cores. And you don't have to do anything to reap this because the Haskell runtime is non-blocking. The only other (relatively common) language that has non-blocking IO built into the runtime is Erlang.

This post affirms for me, yet again, that the people who think Node.js represents a notable, significant improvement over current tools are noobs.

I didn't know they were competing. Reminds me of Rails v Django articles.

Maybe I'm being to generous in reading it, but I took the original question as "I'm familiar with Node and the problems it solves. I'm interested in Haskell; can it solve those same problems?"

The Church-Turing thesis says that problems solvable by a UTM are also solvable by lambda-calculus. So yes, any problem that can be solved in javascript can also be solved in haskell.

This is a flawed argument, the problems that higher level languages solve are those of organisation and program structure.

Which, while true, is about as helpful as answering "Yes." to the question "Do you know what time it is?".

Which is about as helpful an answer as you can get when you ask if a 'language' (haskell) solves the same problems as an 'API' (node)

No, it's about as unhelpful an answer as you can get. Why would you even bother?

Would you answer him the same way in person if he came to a Haskell meetup?

This is not really about the Haskell language, but rather the GHC compiler and runtime that implement the language.

What the GHC runtime does is somewhat comparable to what Node.js does.

By the same token, any problem that can be solved in javascript can also be solved in CSS3+HTML or brainfuck.

...time to get to work on that framework...

CSS3 + HTML isn't Turing complete

http://lambda-the-ultimate.org/node/4222

Actually, it is: https://github.com/elitheeli/oddities/blob/master/rule110-gr...

Aren't Rails and Django competing? I'd hope so...

Reply on news.ycombinator.com