Hacker News Reader

Comment by jasonlotito | Hacker News Reader

So, this article contains a serious issue.

What is the prefetch value for RabbitMQ mean? > The value defines the max number of unacknowledged deliveries that are permitted on a channel.

From the Article: > Turns out each RabbitMQ consumer was prefetching the next message (job) when it picked up the current one.

that's a prefetch count of 2.

The first message is unacknowledged, and if you have a prefetch count of 1, you'll only get 1 message because you've set the maximum number of unacknowledged messages to 1.

So, I'm curious what the actual issue is. I'm sure someone checked things, and I'm sure they saw something, but this isn't right.

tl;dr: prefetch count of 1 only gets one message, it doesn't get one message, and then a second.

Note: I didn't test this, so there could be some weird issue, or the documentation is wrong, but I've never seen this as an issue in all the years I've used RabbitMQ.

View on HN

That's my thinking as well. Seems like they're not using the tool correctly and didn't read the documentation. Oh well, let's switch to Postgres because "reasons". And now to get the features of a queuing system, you have to build it yourself. Little bit of Not Invented Here syndrome it sounds like.

I mean, at some point you do have to write code. Either

1.) You (hopefully) know a bit about how your DB works, what the workload is, what your tasks are. You also (hopefully) know a bit about SQL and Postgres. So you learn a bit more and build upon that knowledge, and implement the queue there (which comes with other benefits).

2.) You learn about a new package, deal with how to set that up, and how to integrate it with your existing database (including how tasks get translated between the queue and your existing DB). This also increases your maintenance and deployment burden, and now developers need to know not only about your DB, but the queueing package as well.

There are certainly cases where #2 makes sense. But writing off #1 as NIH often leads to horrifically over-engineered software stacks, when 10s/few hundred lines of code will suffice.

> Seems like they're not using the tool correctly

To their defense they openly admit this

> and didn't read the documentation.

They clearly did, they even linked to it.

But, as a sibling comment mentioned, they could probably got away with a basic get instead.

As documentation writers (and often that should be read "as developers") it is our task to make our users fall into the success pit - and stay there.

Unfortunately, by not considering this, thousands of hours get lost every year and lots of potential users leave because they were somehow (Google, random blogs, our documentation) led into the hard path.

My favourite example is how, for years, if you tried to find out how to use an image in a Java application you would end up with the documentation for an abstract class 2dgraphics or something, while the easy way was to use an Icon.

The RabbitMQ stuff seems pretty straightforward:

Channel prefetch:

https://www.rabbitmq.com/confirms.html

"Once the number reaches the configured count, RabbitMQ will stop delivering more messages on the channel unless at least one of the outstanding ones is acknowledged"

consumer prefetch:

https://www.rabbitmq.com/consumer-prefetch.html

So a prefetch count of 1 = 1 un-ACKed message -> what they want

The article unintentionally communicated more about the engineering team than the subject.

btw, publisher confirms used in conjunction with prefetch setting can allow for flow control within a very well behaved band.

People run into issues with Rabbit for two reasons. You noted one (they are documentation averse), and number is two is mistaking a message queue for a distributed log. Rabbit does -not- like holding on to messages. Performance will crater if you treat it as a lake. It is a river

great point and same is true for postgres! where I'm at DBAs rip people's legs off for idle in transaction over a few seconds

Agreed !

If prefetch was the issue; they could have even used AMQP's basic.get - https://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.get

This is most likely correct. They didn't realize that consumers always prefetch, and the minimum is 1. Answered here: https://stackoverflow.com/questions/39699727/what-is-the-dif...

I'm wondering if they made the mistake of acknowledging the message before the processing was done. From the article it sounds like their jobs take a long time to run so they may have acked the message to stop RabbitMQ from delivering the message to another worker for retry but IIRC there is a setting that allows you to extend the "lease" time on a message before retry.

Maybe this is a case of an engineer who just wanted to put 'implementing a queue using postqres' on their resume.

> took half a day to implement + test

so seems like there are maybe 2 or 3 services using rabbitmq

smells funny to me.

Reply on news.ycombinator.com