Asynchronous Databases: DBSlayer and HTTP | Hacker News Reader

Asynchronous Databases: DBSlayer and HTTP

igvita.com

26 pointsigrigorik16 comments

FTA:

The problem is that dreaded dynamic request which cuts through the cache and requires a blocking database query, which has an unfortunate side effect of locking up the resources on your application server as well.

Come again? I'm not aware of any database technology where a synchronous request on one socket locks up the whole application server. What on earth are they talking about here? Sockets are cheap. Thread stacks are cheap. Just make the call and let the database sort out the locking and synchronization.

I don't even see how the linked library fixes the problem anyway. It just moves the blocking socket out of the web/app server and into a chunk of middleware that manages its own database connections.

Thread stacks in your web stack are not necessarily cheap.

That's mostly a myth. Yes, you have to worry about what you put on those stacks. But 2-3 pages per blocking call is noise even on low end servers these days. At my day job, we have an application (not a web app, but something similar architecturally) which routinely has 4-700 threads blocked on I/O on a 2G machine. The load average scares the IT guys, but the application works very smoothly and scales much better than anything using select() or poll() would (CPU overhead checking 700 file descriptors isn't trivial). And the code is cleaner to boot.

The "don't use threads for I/O" notion comes out of the early 90's, when OS schedulers were more primitive and memory was more scarce.

First, there's more to it than simply whether the context switching and stack reserves are going to kill you when you have 700 blocking threads (and note that 700 is not a crazy high number, especially for async). Leaving aside correctness, multithreading also incurs synchronization overhead; set aside the "GIL" stuff and you're still left with the fact that in the real world, you will still wind up serializing your program on one of your own data structures.

Second, the "don't use threads for I/O" notion comes from the fact that no matter how much compute you burn on I/O, the data isn't coming any faster, and in I/O bound programs, it's more efficient to use the I/O scheduler (kqueue and epoll in crazy programs, select in everything else) than the context scheduler.

Third, you're about 10 years behind the times yourself if you think that the CPU is going to strangle itself checking 700 file descriptors. Although I don't even think a 1997 async network program is going to lose to a 2008 threaded network program over 700 connections, it isn't even a contest in 2008 vs. 2008.

I am happy to race you. =)

Regardless, I wrote an async MySQL driver because you can't demand-thread per-packet network code, and I wanted to dump stuff to a database. Some of us will thread when threading makes a program cleaner and simpler, and run everything async when threading becomes an obstacle course.

Tom, you're not making any sense here. You stated that "threads weren't cheap". I said this really wasn't true anymore for this problem area (mostly-blocking network I/O) and mentioned an application I'd written that shows exactly this over a load regime very close to what you'd see with a heavily-contested web application database.

Now you're talking about stuff like synchronization overhead, which wasn't at issue: synchronous access to a few hundred separate file descriptors (one per thread) obviously doesn't need to synchronize anything. You're stating some stuff of questionable veracity (select/poll certainly do scale poorly once you get past a few hundred descriptors -- just check the kqueue/epoll justification documents for copious benchmarks to that effect).

And you're even making up some, er, interesting new terms: what on earth is an "I/O scheduler" as distinct from a "context scheduler"? Usually when you use the former term, you're talking about the block device request scheduler (elevator algorithm, etc...) which (1) isn't involved here as we're talking about balancing network I/O and (2) is invoked in the same way for local I/O regardless of whether you're doing I/O via an async request for 700 blocks or via 700 synchronous read() calls from separate threads.

And you capped it all off with a few ad hominems that I honestly don't think are appropriate on this site, at least in a technical context. Stop flaming.

I'll say it once more: threads are cheap in this regime. The linked article is a hack to get around the problems of database access from monolithic, poorly threaded language interpreters. It's not a "performance enhancement" in any meaningful way. Even a shared-nothing web app architecture a-la news.arc is likely to do better than an extra hop through this thing.

If you can be specific about how what I said was uncivil, I will apologize for offending you. I think you're wrong, but I don't think you're crazy or stupid.

You're right that we started talking about something very specific (the memory costs of stacks for 700 threads) and I quickly generalized (to the performance of threading versus async code). That's a fair critique. In my defense, the performance difference between threaded code and async code is very relevant to this article.

Here are my points:

* It is not a "myth", as you say, that async network code scales better than most threaded network code.

* Your anecdote about 700 concurrently served connections probably won't cause select(2) to break a sweat, let alone kqueue/epoll.

* It's horribly unfair to tar async code with select(2)'s performance, because performant applications use kqueue or epoll to replace it. Both resolve the scaling problem you're alluding to.

* It is perhaps weird that I see a similarity between a thread scheduler, which switches CPU contexts on a timer, and a select loop, which switches them nonpreemptively on I/O events. I retain the right to say that event loops are an instance of the "scheduler" problem; if you think that's crazy, read the papers on the MIT Click modular router. In fact, do that anyways; they're great.

ajross: Ruby / Python doesn't have a great threading model (GIL in both cases is the culprit). Hence, if you have an app-server which is single threaded (event loop), or have other locks around servicing requests (like Rails Dispatcher), when the server makes the DB call, it effectively becomes unresponsive until the data is fetched.

You're absolutely right, sockets are cheap (if you can defer to kernel's select()) which is exactly what we're doing here. Instead of doing blocking IO, we convert the DB call into an asynchronous call.

What's a blocking database query? Readers don't block writers and writers don't block readers in any sane database in the last 20 years.

The situation you can see is unexpectedly long-running queries (say distributed transactions) causing the middle tier's connection pool to be exhausted, but that's not the same thing at all.

Their point is that it's a synchronous query. Just like a blocking read from a disk file, you have to wait until the answer is available before issuing another one. If you have a poorly threaded app server (which seems to be the use case here), that's a needless performance bottleneck.

If you're trying not to spawn threads at all, blocking database connections are also a dealbreaker.

Fantastic. Instead of facing up to the crippling inadequacies of Ruby's single threaded implementation, we get a mad scientist solution.

I swear Ruby attracts bad thinking like a magnet.

I don't think that's the case at all. The problem is exactly the same in any language - which is why we've seen a multitude of Java / Perl / Python / Ruby / C drivers developed to do the exact same thing.

As someone has already pointed out, it's not about the threading, it's about turning a blocking request, into an asynchronous one. (Deferring work until you have the data to perform meaningful actions with it).

I think the reason this is a bad idea is that in order to turn a blocking operation into an async one, you're adding a totally new transport layer (HTTP) and data representation (JSON), as well as middleware. That means a significant performance hit, and a lot of needless complexity (and potential for failure).

It would be much simpler to either make the database driver itself non-blocking (Postgres' client library supports async queries, for instance), or use a dedicated pool of threads to perform blocking DB operations. It might be that Ruby's threading is too broken for the latter to be feasible, but it still makes using DBSlayer just to get async DB operations needless overkill.

I actually tend to agree; Ruby's equivalent to Python's "Twisted" is EventMachine --- a respectable event loop library --- and there is a pernicious meme there that one of the things we should be doing with the event loop is yet-another-nonpreemptive-thread-scheduler.

There's nothing wrong with an app stack that is async up through the database client. Async is good. There are things that are wrong with app stacks that are entirely threaded, but we can have that argument somewhere else. But there is something wrong with the idea that we can have the best of both worlds, if we just try to make async code look like threaded code.

what's the use case for this? are you using it for aiderss?

mtw, haven't deployed it in production, but playing with it. The nice thing about DBSlayer is it provides the round-robin balancing as well! (Read from multiple slaves, transparently)

Reply on news.ycombinator.com