Ask HN: Is There a “Smart” Message Queue? | Hacker News Reader

Ask HN: Is There a “Smart” Message Queue?

3 pointsbgdam1 comment

I'm currently using RabbitMQ for a variety of message queuing needs, and I'm running into a specific issue repeatedly.

If I push 10 messages to a queue, and say 5 of them have the same 'key' parameter, then I want to automatically discard the previous messages still in the queue with the same key.

Is there any message queue which already has this baked into it out of the box?

View on HN

Architecturally, the idea of discarding duplicate messages from a queue is:

  queue -> parser -> queue

Discarding duplicate messages is a piece of business logic. The parser could be as simple as a hashmap/dictionary or bloom filter that emits or does not emit the message based on the selected criteria.

This is a case where I like the idea of Unix pipes and where I might think in terms of log data structures rather than queues. Your mileage may vary.

Good luck.

Reply on news.ycombinator.com