Hacker News Reader

Comment by stolsvik | Hacker News Reader

Well. My point is that messaging inherently have all these features, without needing any other tooling.

The combination of transactionality and retrying is hard to achieve with REST, don't you think? It is actually pretty mesmerizing how our system handles screwups like a database going down, or some nodes crashing, or pretty much any failure: The flows might stop up for a few moments, but once things are back in place, all the flows just complete as if nothing happened. I shudder when thinking of how we would have handled such failures if we used sync processing.

The one big deal is the concept of "state is on the wire": The process/flow "lives in the message" - not as a transient memory-bound concept on the stack of a thread.

View on HN

> inherently have all these features, without needing any other tooling.

Providing your queuing service magically scales. and you can just scale that infinitely

If your queue is the bottle neck (yes it happens) how do you know there are more nodes getting added to it? How do you rebalance the topic over multiple new nodes in the queue?

> I shudder when thinking of how we would have handled such failures if we used sync processing.

easily. oh look i got an error and retried a few seconds later and it went to a new isolated backend running separately somewhere else or after it started again.

> If your queue is the bottle neck (yes it happens) how do you know there are more nodes getting added to it? How do you rebalance the topic over multiple new nodes in the queue?

In a message queuing system, there is no rebalancing - you just add nodes, which then also get messages, typically in a round-robin fashion. This is not Kafka.

But of course, you can max out your message broker. As you can with a network switch for that matter.

> oh look i got an error and retried a few seconds later

So now you need a retrying mechanism. Which is provided as a basic feature of a messaging system. And with the transactionality, you do not get the problem of wondering whether the first attempt went halfway through, or not.

> In a message queuing system, there is no rebalancing - you just add nodes, which then also get messages, typically in a round-robin fashion.

And hope that the DNS\config propagates to the calling client and that the library can reliably add and remove nodes.

> Retrying

Wanna bet it's a built in feature in your messaging system?

Cause the whole publish op can fail at which point you have to retry sending it. or it published but you lost the ack (seen with AQ more than once) so now you send that message again and hope you remembered that whole duplicate message handling

> And hope that the DNS\config propagates to the calling client and that the library can reliably add and remove nodes.

This is not how it works. A messaging broker client connects to the message broker. The client then creates receivers for one or multiple queues. If the broker has multiple clients receiving from the same queue, it uses a round-robin dispatch to the clients.

I fail to see how DNS or config, or "reliably add and remove" factors in here?

> Wanna bet it's a built in feature in your messaging system?

Yes?

This is not using pub/sub, but queuing. (There is an option of using publish/subscribe, but that is only meant for special cases like updating a GUI, or "broadcasting" an event like "invalidate cache for user X")

Message Queuing can be transactional. Mats leverages this. I have written about it multiple places both on this HN discussion, and on the webpage.

Reply on news.ycombinator.com