Comment by tptacek | Hacker News Reader

Comment by tptacek | Hacker News Reader

As luck would have it, I'm sitting here writing a fuzzer for a trading protocol in EventMachine as we speak.

Ruby and async network programming go together like gin and vermouth. It's one of the reasons I'm so happy with Ruby's block/lambda support, and so irritated with Python's: many of the basic programming idioms for evented network code have direct support in the Ruby language.

I've done a fair bit of Twisted Python programming in the past, and I find EventMachine to be far more intuitive and flexible. On the other hand, EventMachine will crash on me once in a blue moon, and it swallows exceptions, which is extremely frustrating. They're both great packages.

The big problem that still needs solving for EventMachine is evented adapters for an ORM (any ORM). You've always been able to talk to MySQL over an evented socket, but that's clumsy compared to DataMapper.

I am still totally not sold that Ruby threads are tenable, so a lot of the "advanced" stuff in EventMachine does nothing for me. It's great even without it.

I am still totally not sold that Ruby threads are tenable, so a lot of the "advanced" stuff in EventMachine does nothing for me.

FWIW, threads based on an event backend are a solved problem in Perl. The Coro library is what EM is trying to be; they should really take a look. ("Oh but it's PERL!!!!" Yeah, and it actually works.)

Maybe, but part of the value of async programming is never having to thread, so none of this really lights me up.

Honest question: is there a purely event-driven I/O framework (no threads anywhere)?

The one that a lot of people have gotten excited about recently is Node.js and that uses threads for file I/O (libeio).

EventMachine doesn't thread unless you ask it to. I use EventMachine in Ruby and libevent in C with nary a mutex in sight.

I see. My (wrong) understanding was that EM used a thread pool to do file I/O like Node.js. It appears to add the file descriptor to the event loop's watch list.

> Honest question: is there a purely event-driven I/O framework (no threads anywhere)?

You might look into a continuation-based framework. With continuations, you can do purely event-driven IO written in a direct style: the framework can grab the "callback" itself by snagging the continuation of its slow IO functions.

You still run into the same problems with blocking (non-IO, or non-framework) code taking up more than its fair share of time, however.

http://github.com/lifo/cramp

It works best in Thin (EventMachine) or Rainbows! (Fibers).

If you are doing I/O you should use NeverBlock (http://www.espace.com.eg/neverblock/) they've even implemented a MySQL driver, the best part is they're 'drop-in' so you don't have to learn a new API, they just hide the async calls from you.

Fibers I think are closer to Erlang style than the event loop, and much more straight forward to program.

What you you think a "thread" is? A thread of execution? An OS thread? A lightweight thread?

Without defining thread, your question is pretty close to meaningless.

Coro basically allows you to write:

   foo();
   bar();

instead of:

   foo( cb => sub { bar() } );

Behind the scenes, it's basically the same thing. It's cooperative so your code does not get preempted where you don't want it to be; the locking is implicit.

But you can switch to preemption and explicit locking if you desire. At that point, IMO, it's time to ditch Perl / Python / Ruby and use a language with a sane STM implementation, because locks suck.

I'll admit to being a major Perl hater. However, this Coro library looks great! Perl threads are horrible pieces of trash, and the Coro library author not only admits this but does something about it by implementing threads the way nature intended them to be. If I'm ever forced to code in Perl again and need or want threads, I will turn to this library first.

The way the Coro author implements Perl threads, by the way, sounds similar to how MacRuby and Rubinius are attempting to remove the GIL in ruby by copying interpreter state so that multiple OS threads can run simultaneously.

edit: after reading further, it turns out these threads are cooperative in that only one can run at a time. Also, it looks like they act more like Ruby fibers in that one must give up control explicitly or make a blocking call that implicitly gives up control. I'd rather the core scheduler be able to preempt threads directly. What if one of the threads takes a long time or ends up in an infinite loop?

"This module collection manages continuations in general, most often in the form of cooperative threads (also called coros, or simply "coro" in the documentation). They are similar to kernel threads but don't (in general) run in parallel at the same time even on SMP machines"

They can be automatically preempted if you desire, see: http://github.com/nothingmuch/coro/commit/54902809d8d3ad054d...

Usually people dismiss Coro because they want to take advantage of multiple cores without running multiple processes. But the reality is that you'll see a maximum 8x speedup if you run 8 OS threads, but if you rewrite your critical section in Haskell, you'll see a 50x speedup. So really , Coros are exactly the right abstraction for "scripting languages". IO performance is great, and expressiveness is great. Speed can be gained in other ways.

I agree entirely about not needing threads in a scripting language like perl for CPU performance reasons, and being able to configure the threads to be scheduled automatically is a big win in my book. Thanks for letting me know about the library!

How does the Perl Coro library compare to coroutines in Lua or Scheme? (It's been almost a decade (!) since I used Perl really actively, and it seems to have changed quite a bit.)

Reply on news.ycombinator.com