Hacker News Reader

Comment by mrfusion | Hacker News Reader

What's so special about a job queue that it can't use an rdbms? I never understood that.

One reason: Most RDBMS's aren't designed for "waiting for newly queued items". You literally have to have each worker poll at some time interval for new items: `SELECT * FROM jobs WHERE status = 'new'`, transactionally claim the job, and if a worker dies, un-claim it. It's not that it's not possible to use an RDBMS and work around these problems, but generally, it won't lead to the most scalable and robust solution for the problem. Message brokers / AMQP are better designed for this problem set. If you have a small site, with a low volume queue, your RDBMS might be just fine though.

This is not the case if you're using Postgres, which will signal/notify listening clients, instead of having the clients poll for changes.

are you asking why RabbitMQ exists?

Yeah, or any documentation that explains the fundamental differences. I get that it's required and I can see the advantages for large scale operations but I never understood why you can run a simpel DB backend for small scale setup.

Just trying to learn here!

You can use relation database but sane implementation is ugly and doesn't scale. There is little reason to choose bunch complexity and unpredictable behavior over dedicated queue dependency. Sorry, i don't have any good material on the topic. Maybe you can use this to see how RDBMS implementation looks and what are the consequences. https://brandur.org/postgres-queues

Reply on news.ycombinator.com