Comment by silentbicycle | Hacker News Reader

Comment by silentbicycle | Hacker News Reader

Parent Full thread

To me, it seems the real weakness of node.js is having to manage asynchronous control flow yourself, at all. It grows exponentially more ugly as the system increases in size. I really like continuation-passing-style as a technique (especially as a compiler IR), but doing that stuff by hand is so 70s.

I got pretty far along writing a similar system in Lua (because, hey, event-driven systems are pretty nice), and unlike Javascript, Lua has well-implemented coroutines - they singlehandedly eliminate a LOT of the ugly control flow management. Still, most libraries are blocking* . I eventually decided that, for small systems, an event-loop framework wasn't necessary in Lua (it's easy enough to just do it from scratch!), and for larger systems, I was better off using Erlang, which has the ideal foundation for that sort of system. In particular, I'm really not clear how much error-handling node.js does; that was what convinced me that Erlang's approach is refreshingly sane. When I started writing process-supervision and hot update code in Lua, I realized I was just re-implementing what Erlang does best.

* Which is a funny objection, like being mad that most libraries default to using decimal numbers rather than octal. It must be a conspiracy!

See also: http://news.ycombinator.com/item?id=2150800 and http://news.ycombinator.com/item?id=1304599

  > Still, most libraries are blocking*
  > * Which is a funny objection, like being 
  > mad that most libraries default to using
  > decimal numbers rather than octal. It must
  > be a conspiracy!

Actually, there _is_ a distinction here: It is ridiculously easy to implement blocking semantics on top of non-blocking semantics - in some pseudo code:

  event = do_async();
  pause_until_completion_or_error(event)

Whereas implementing non-blocking semantics using an underlying blocking implementation is quite, but not entirely, like banging your head against the wall. repeatedly. Essentially, you cannot avoid threads, shared mutable state, and a lot of other problems.

This may surprise some, but implementing blocking semantics on top of non-blocking semantics in node.js is impossible.

Thanks for this, you've nailed the problem.

I'm looking for a language that can do everything that Node.js does but also solves this control flow problem, but is also fast and has support for OpenCL and beefy math libraries.

I don't find Erlang to be performant in this regard, so I'm looking at Scala but I'm constantly getting pissed.at.java.packages.and.conventions. I think it also suffers from the same problems as Node.js and doesn't support the async/control-flow magic that Haskell supposedly has.

Perhaps I should consider Haskell after all. It's that or C++.

Take a look at LuaJIT.

I have some early FFI bindings for OpenCL (and OpenGL, glfw) with some simple demos. Not updated recently, but it's done by using stock luajit (take latest), and not writing a binding code at all (just FFI definitions which LuaJIT understands).

You can't do callbacks (yet). That is - you can't rely on "C" function to call you back at lua land. That's why I chose glfw instead of glut or others. There is way to setup your main loop without callbacks.

http://github.com/malkia/luajit-opencl

There is also a coroutine based embedding of Lua/Luajit in nginx if you want to embed into a web server environment (without callbacks). https://github.com/chaoslawful/lua-nginx-module

Perhaps you could try python, numpy, twisted, inline callbacks and one of the python opencl packages.

Inline callbacks are syntactic sugar included recent versions of twisted that make your code look synchronous while still running on the event loop.

With python, you get to use a wealth of libraries, but you need to still be mindful of what is blocking.

Wnen something you need to call is blocking, you can try deferToThread.

Don't worry, you won't need to think too much about race conditions and other concurrency issues with threading since there is the GIL in python. Also, Twisted makes it a cinch.

Good luck!

Haskell (or rather GHC) offers a blocking programming model (e.g. spawn a thread per connection and make blocking reads/writes on the socket) but uses asynchronous I/O in its implementation (one thread uses epoll/kqueue/poll to do the I/O and the CPU bound threads are scheduled on a thread pool).

Maybe, in a couple of years, javascript (or any language like coffescript that compiles to javascript) will fit your needs, especially regarding the control flow problem.

Here's an excerpt of a blog post (http://blog.mozilla.com/dherman/2011/01/30/proper-tail-calls...) from a research engineer at Mozilla Labs who works on the new ECMAScript standard:

  >Having an officially guaranteed tail call mechanism   
  >makes it possible to compile control constructs like 
  >continuations, coroutines, threads, and actors. And, 
  >of course, it’s useful for compiling source languages 
  >with tail calls!

Of course, it is a long way to go until one is able to use these features. So, for now, you are better off to look somewhere else for your specific needs.

You could build the skeleton in Erlang and then call out to Lua. This one works well...

https://github.com/Eonblast/Erlualib

We developed a coroutine-based event-driven networking lib in Lua (not open sourced yet, but it looks like it might change someday). One of the neat thing is that we're able to run that on tiny ARM-based embedded hardware.

I think there are people who did similar stuff for classic PC hardware, and already open-sourced their work; you might want to look at nmap, among others.

The external API is a superset of Luasocket's, so the event loop can remain completely transparent for users if they choose so. Proper coroutine support really rocks!

Reply on news.ycombinator.com