Greenlet-based event I/O Framework for Python | Hacker News Reader

Greenlet-based event I/O Framework for Python

diesel.io

63 pointsareski20 comments

I'd be curious to see this benchmarked against Tornado, Twisted, gevent w/ Flask, and Node.js. Would be fun to see where it fit into the possibilities for the popular frameworks for writing realtime web stuff.

I'm starting to write a realtime backend right now in Golang and am excited to see how it goes. Really enjoying concurrency with Go so far. Would be glad to hear anybodies opinions that have done any realtime backends in Go :)

This was a famous but now dated (2009) socket benchmark of frameworks that involved Diesel http://nichol.as/asynchronous-servers-in-python

I wouldn't look at that. That is 5 years old, and that was a different Diesel framework.

Ok, same name (a cool one, I must admit), and I think same group of people. But, it did concurrency differently. It used to be a yield based API. Which, funny enough, is like Guido's latest Tulip/Async.IO approach. And it seems me they (Diesel) decided against it, and I 100% agree. Having yields in the library code, aside from demos and small examples, doesn't work very well.

I think Diesel creators came to their approach from a practical standpoint. And I think Guido (and other proponents of async yield based IO) came from a purist/theoretical approach. We'll see what will happen in the future. It is hard to tell. But I don't see yields everywhere as a good thing. Just like I don't see callbacks or deferreds everywhere as a good way to do concurrency.

greenlet (the base module gevent and eventlet are based on it) is probably the best thing that happened to Python in its recent history. It is really unfortunate it wasn't adopted as the main concurrency mechanism in the latest version.

"yield from" isn't available on Python 2.7, maybe that's why they switched?

That would have helped a lot, yeah. Also, dealing with exceptions in an intuitive way when you have a "fake" stack you're managing yourself is a drag. The performance of of managing your own stack also sucks.

But probably the biggest practical issue is there is a terrible, common silent mistake pattern, that bit us over and over again:

    self.do_something_async() # it's a generator

What you really wanted to do was to `yield` this so the reactorish element in the frame above you consumes it. Instead, you created a NOOP generator that is never consumed, so whatever work you thought you did, silently.. does nothing.

Anyway, I think `yield from` helps you get around some of this stuff, but I haven't been keeping up with tulip. Which is probably kind of silly since Guido works like 50 feet away from me.

Btw, hi Donovan. :-)

What was tulip (and is now asyncio in 3.4) has a debug flag that you can enable to detect if a coroutine is scheduled but never called.

More info here: https://docs.python.org/3/library/asyncio-dev.html#detect-co...

Thanks for the tip, that's really useful info.

Hi Jamie. That's some really interesting insight, thanks! I'm toying with generators in javascript now and I can see forgetting the yield being an issue. The other reply to your comment seems to suggest there is a solution though, I'll have to look into that.

Greenlet giving great exception stacks sure is nice, and greenlet's performance is REALLY nice.

(Can you believe it has been almost 10 years??)

PyOhio 2013 presentation http://pyvideo.org/video/2278/diesel-simple-and-scalable-net...

This is old. Is it still maintained after Google acquired Bump?

Not much recent activity: https://github.com/jamwt/diesel/graphs/contributors

Dowski still works with it routinely at GOOG. I'm knee deep in golang at Dropbox myself these days, so I don't get around to working with diesel much.

A little bit meta, but: The funniest part of sites like these with real-time chats getting posted, is seeing how prim and proper our comments on HN are compared to what we're all like in a chat box.

Let's hope this never gets popular: http://hnlivecomments.pex2.jp/

a trip into the past :P

There's something adorably quaint about it. Multiple people can use the same name (and somebody's exploiting that to great success right now), but stuff like this still happens:

    <CSS DB>I'm waiting for HTTP 2
    <111>why?
    <CSS DB>Just waiting
    <111>Let's wait together.
    <CSS DB>sounds nice .. 2 dude
    <111>after sucking such shit together, I see you all as a family
    <CSS DB>thank you for being that deep
    <CSS DB>I will save you wait for me where Adele is rolling
    <111>somebody has to

Maybe I'm jaded, but this doesn't seem to happen in places where you have to worry about your reputation. And maybe you lose something by that.

Or maybe I just don't get out enough.

I can imagined a funny scenario, where someone records and links it to the actual names or maybe just HN handles. Then one day (maybe tomorrow) reveals those links.

Things like "HN handle Twisol said -- 'after sucking such shit together, I see you all as a family'. And then people look and see your other posts about serious topics as branch prediction in compilers and monetization strategies in start-ups.

No Python3? I can't really find anything on supported versions.

EDIT: Looks like it's only for >=2.6 https://github.com/jamwt/diesel/blob/master/setup.py#L3

That's unfortunate because I need a clean event framework for Python3.

gevent should have py3 support soonish since dropbox put their money behind it: https://github.com/surfly/gevent/issues/38

You can use gevent in a similar manner to this - monkeypatching is not required

asyncio - 3.4 has event driven built in goodness trollius - a backport of the 3.4 asyncio library for python 2.7

also take a look at Pulsar its an event driven / Actor model concurrency framework that supports 2.7 and up using asyncio and trollius

this is good but I'm having trouble understanding the code on the first page.

also some sort of benchmark would be very interesting.

we desperately need something like meteor.js in python. imagine if you could write python on client and server and end up with something like meteor.js app

Reply on news.ycombinator.com