Hacker News Reader

Comment by indymike | Hacker News Reader

I'm putting this one on my new hire required reading list. So often people end up re-inventing message queues and not even noticing they are doing it.

View on HN

I've bumped into this recently. Someone waved their hand and said that x queue system didn't work for their specific use case. so they spent an innovation token on making a slow unmonitored queuing system. unsurprisingly the project is late, and it only now has started to work on the bits its supposed to handle. (three months after the public announcement)

When I asked for the rationale, they said that x queue system didn't do ordered delivery. The queue system literally has the word "ordered" in it.

Its not really their fault, they project manager should have stopped it. However its still not an excuse to re-invent the wheel.

Yeah, a similar nightmare prompted a lot of this reasearch. A colleague would argue for what felt like hours against SQS because it was “random” ordering, and kept demanding we use something that had strict FIFO. Despite the clients who placed the jobs not being affected by or caring about execution order. The worst part was that we had paid a shit ton of money for many globally distributed executors to increase throughout - I simply didn’t have the vocabulary to explain how asking for strict FIFO would mean only one executor would be running at a time, wasting hundreds of thousands of dollars.

This is along the same lines, but something people don’t often think about when using and naming queues.

https://www.brightball.com/articles/organizing-background-wo...

> people end up re-inventing message queues and not even noticing they are doing it

Amazing how people sucked down this rabbit hole. My work is doing this right now, and its driving me mad as there are well supported alternatives out there. But I suppose there is something intrinsically fun about building a complex communications system.

I've experienced excellent results building distributed systems where the nodes talk to each over via raw udp. often, in the context of message queues, there is much talk about delivery guarantees, but in many contexts the same goals can be accomplished by designing the system so it can handle lost messages. for example, rather than a node sending a single message notifying that it has started work on something, it sends a message once a second, saying, 'I am currently working on x'. is that reinventing the wheel? to me it just seems much simpler.

Interesting. This is actually sort of a distillation of all the new hire training / discussions I’ve done. It’s as much a “things I wish someone had told me 15 years ago” thing too.

Would you be willing to share the list with us?

Hmm. I really should find time to get my personal site back online. I took it down a month ago to re-face it and life has intervened.

You could just dump it in a gist

Here it is: https://mikeseidle.com/2021/01/03/my-developer-new-hire-read...

Reply on news.ycombinator.com