Threads (in Ruby): Enough Already
yehudakatz.com
yehudakatz.com
It's also not a very compelling argument for threading to say "web programs don't share a lot of state, so you don't have to worry about synchronization". If all you do are CRUD apps, you can indeed punt scaling to the database. That doesn't mean threads are more effective than events; it means you made concurrency and synchronization someone else's problem. There's nothing wrong with that, but it's not a convincing demonstration of threading.
Truly? To my understanding, event loops require inversion of control (and likely callbacks and broken exception handling). This is a large cost that requires a benefit to be worth it. I understand that benefit to be: you don't have to deal with threads (or bad implementations of such).
This blog post comes to mind: http://www.unlimitednovelty.com/2010/08/multithreaded-rails-...
Async code isn't "likely" to require callbacks; it will almost certainly involve callbacks, those being a central design feature of async code. You should play with the idea for a bit before forming an opinion about it.
I don't know what you mean by "broken exception handling". This may be a Rails-ism I'm unfamiliar with. I'm very familiar with Rails (we ship a fairly large product built on it), but I've never tried to shoehorn async I/O into it; like Twitter and presumably every other large site, we do Rails on the front-end/UI and fast things on the backend, in our case with EventMachine.
Like I said: I'm not arguing that Rails threading is bad, or even that Rails should have better async support. If I cared that much about the performance of my front end, I probably wouldn't be serving requests directly off Rails. Rails developers may very well be better off with threads. But that fact has little to do with the merits of threading and async as concepts.
Take the following Python-esque (but not Python) psuedo-code in a threaded language:
try:
header = read(socket, header_size)
if has_short_flag(header):
return read_until_closed(socket)
else:
handle_message(header, socket)
except SocketClosedException:
log("lost socket while processing headers in packet")
This is one of the reasons I've spent the last few years fleeing event-based code towards things like Erlang, rather than running towards it; I've been doing event-based code for non-trivial work and things well beyond "demos" and the plumbing just explodes in your face if you want to build actually-robust software where simply crashing in the middle of a request isn't acceptable. Despite my best application of good coding practices and refactoring you still can't get close to the simplicity of something like Erlang code.
(By the way, if you are stuck in evented land, one of the things I have learned the hard way is that anywhere your evented API has an error callback, you absolutely must provide one that does something sensible. I now always wrap such APIs in another layer of API that does nothing but crash as soon as possible if no error callback is provided. If you can't figure out what your error callback for a given such call should be, that's your design trying to tell you something's wrong.)
That said, your (admittedly hypothetical) example is hideously broken. In EventMachine, and written properly:
def receive_data(buf)
@buf << buf
while have_complete_request?
begin
send_data(handle_next_request)
rescue => e
log(e)
end
end
end
The mistake you made (and it's common to a lot of evented code) is in driving the entire system off raw events. Don't do that. Buffer, to create natural functional decomposition.
Evented code is rarely as simple as synchronous code, but there's no reason it has to be needlessly choppy.
That said, I think this design is overvalued. Yes, it's true, you (probably) can't always wrap an entire request's processing in a single exception handler in any evented Ruby library I know about. But I wouldn't wrap an entire request handler in a single exception handler in any case! If I was preparing to deal with a database exception, I'd wrap the code making the database call in an handler for the database exception. If I was preparing for a filesystem exception, &c &c &c.
Incidentally, I've been doing very, very, very large scale evented systems for going on about 10 years now (large scale: every connection traversing tier 1 ISP backbones), and, sorry, this stuff has never blown up on me. I may have been shielded from exception drama by working in C/C++, where exceptions are not the norm. I was a thread guy before that. Threads definitely did blow up on me, a lot.
See, I wouldn't even call that "evented code" in the way that people are using the term, regardless of what it says on the tin, precisely because you aren't losing the stack frame here and can still catch exceptions and such. Evented to my mind is something like Node.js is when you have to chop up your code manually. At least, I've never seen anybody demo Node.js code that isn't chopped up manually and I am at a loss as to what features of Javascript would let you translate that Ruby snippet directly without losing something fundamental about the stack.
Under the hood, everything's event-based (with optional preemptive multitasking), there's just varying levels of compiler optimization that affects how much you have to do manually and how much you have to worry about it. The inner event loop of Erlang and the inner event loop of Node.js and in fact the inner event loop of just about anything nowadays looks pretty much the same.
That's not the way in which I say evented code blows up. If you can write like that, it doesn't blow up, because you don't have to sit there and basically manually implement your own stack handling if you want anything like that sort sane exception handling, it all just works.
Since this is a terminology issue there is, as always, grey areas, but since I mostly use the term evented in the context of the Node.js hype I tend to use it that way. I've been doing stuff like your snippet for a while too and it hasn't blown up on me either, which is why I'm so down on the style of coding Node.js entails, which does.
The point of my snippet is not that that is a brilliant choice intrinsically, the point is that you don't have the choice and end up implementing anything like that manually.
Javascript's a very nice, dynamic language in most ways but its functions are very, very weak. They'll get better in the next version of ECMAScript, though. (Node.js will probably benefit from it.)
try:
something = some_io_that_needs_an_event_callback()
if do_something_with(something):
more_io_with_callback()
some_other_callback_thingy()
else:
yet_other_io()
some_other_io()
except AnyException:
do_something_sane()
Your code block says it is available to you in Ruby. There's no reason why it shouldn't be; it's not a hard compiler transform to manage that. Taking code that looks like a "thread" but compiling it into events and using something-like-continuations to keep the stack together is on the order of a homework problem. Based on what I know about Ruby, the responsibility of those pieces is split up differently but all the pieces must be there. But it's not something Javascript can do (quite yet, and based on what I've seen in Python the 'yield' approach has its own issues).
Nope. Coroutines cut that gordian knot. Inversion of control is for when you want to make code verbose and hard to reason about!
I have a partially complete node.js-esque system in Lua (Javascript's smart brother, which has efficient and well-integrated coroutines), but I may not finish it for a while, at this point - I've gotten pulled into Erlang instead (and life, in general). I kept feeling like I was reimplementing Erlang, anyways. (My LuaSocket.select-based backend works, but my high-scalabilityTM libev backed is on hiatus, and the usual select vs. epoll/kqueue trade-offs apply.)
I'm making an argument about how threads are used (in real life) in web development, an area where it's trivial to make concurrency and synchronization someone else's problem. Despite this, I've heard a number of hypesters throw around the idea that this scenario is an example of whether threading fails and moving to async is required.
I agree with you that this is a weak argument, and I hope to see people understand better the difference between:
a) an application that NEEDS to handle huge amounts of concurrent users (because most of them are idle for most of their lives), and
b) an application that spends a non-trivial amount of time using the CPU, and therefore does not need more than a few threads to fully utilize the CPU
There are different cases, and while those of us with a good grasp of the subject understand the difference, a lot of people have conflated the two ideas, and then further conflated the problems of thread synchronization in these cases as well.
It was things like the comparison to Node (which I don't use) and the comment about how well async had worked for you in browser js --- which implicitly somewhat demerited serverside async --- that made me think you might have been reaching for something more ambitious with this post.
I explicitly referenced a chat server (lots of concurrent, mostly-idle requests) as a case where I'd personally use an async solution.
There are certainly middle-ground cases where the question is muddier (and more religious, likely), and I just wanted to set the record straight that Rails itself is not really in the middle. I'm glad that you agree :)
Isn't this an issue with both models? Shared state is shared state, regardless of whether you use threads or an evented model. Unless you're only running on 1 CPU.
This goes out the window when you start forking processes and using shared memory, but at least then you're default-private instead of default-shared.
Forking seems...a little weird to me coming from my previous Windows background.
Forked process in ruby have posix semantics, ie, NOT shared memory. Linux uses Copy-On-Write pages to conserve the amount of memory copying when the new process is created.
As I commented earlier, Ruby Enterprise Edition (made by the same guys as Passenger) ships an alternate GC that doesn't actually write to the memory space of the original objects in order to mark them.
As a result, some memory (but not necessarily as much as you'd expect) can be shared between processes forked from the same parent and running on multiple cores. Sort of a middle ground between totally separate processes and shared memory via non-GIL'ed threads.
The Brightbox team tried to tackle the related problem for moving to Ruby 1.9 with a crowdsourced library review site: http://isitruby19.com/ Does anyone know of a similar site where people can report whether libraries are thread safe? (The code for Is It Ruby 1.9 is available on github - http://github.com/brightbox/isitruby19 - so it would be easy enough to set up a clone, but I can't commit to maintaining such a thing.)
The way it works is that the author can specify whether or not he thinks it's threadsafe (yes/no/maybe), which is then verified by users who can specify whether they agree. If a user marks a plugin as not-threadsafe (or not-Rails 3, or not-JRuby, or not-Ruby 1.9), the author has 7 days to help the user come around before it sticks.
So far, there are 60 plugins marked as threadsafe (which means that either the author said "yes" and nobody disagreed, or the author said "maybe" and all the votes so far say yes).
I found that the postgres gem, the most immediately important for me, isn't listed. Do I need to be the gem author/maintainer to register a gem, or can I just add it?
As a psql fan, it makes me wonder what the rails world would be like had dhh had picked psql from the start.
If you use JRuby, you'll need one Ruby process per machine (for N cores), managed by the JVM.
For boxes with a lot of cores, JRuby's larger memory footprint is overtaken by the ability to share that memory across a number of cores.
This story is also somewhat complicated by Ruby Enterprise Edition, which adds copy-on-write semantics to Ruby's GC, and is built by the same guys as Passenger, making it possible to share SOME memory between processes.
With all that said, we're really talking about marginal amounts of RAM. The real takeaway is that if you're running 6 processes per core (very common), you're doing something very wrong.
FYI: At some point in the future (1.2?), Rubinius will also be able to run a single Ruby process per machine.
Additionally, even Hotspot typically defaults to their stop-the-world GC. This is because a concurrent GC typically spreads part of the GC time around, ie, performance is slower to reduced GC pause time. But even a concurrent GC typically has to stop all threads at some point to get everything consistent.
Ruby's "stop the world" garbage collector means a process will completely pause during GC, including all threads. You may be better off having twice as many processes at half the ulimit. Fewer threads will be paused at once, and there are fewer objects the VM has to traverse during a GC run.
It's something to investigate when you tune your app.
(This mostly applies to 1.8. I haven't investigated 1.9.)