Hacker News Reader

Comment by PascalW | Hacker News Reader

I don't really understand this sentiment of not needing a queue system. This is not much different from spawning a thread in Java to delay the email sending.

For any serious application you want a job like that to be persisted so you can guarantee it runs even if your application is restarted.

I know that Erlang/Elixir is designed for stateful applications and if you have a cluster and do hot deploys this is less of a problem, but who does that? Most Elixir applications I've seen are deployed as stateless systems just like any Ruby, Python, Node etc systems.

View on HN

Slightly offtopic, but consider a pattern of persisting the state machine of a given task instead of the task queue itself. For example, for a daily email job, you might have three states: UNSENT, BEGIN(begin_time), SENT. Then you can just have a supervisor job that scans your table or whatever and schedules jobs, retries, etc. as needed. Now you don't have to worry about the queue state at all!

Having a queue means you have a distributed system. How do you handle network problems, errors/retries, back pressure? OTP has excellent idiomatic tools for all that and more.

Elixir developers who need a queue generally reach for Rabbit (which any language can use), or something backed in a database like rihanna[1] or honeydew[2]. Rolling your own distributed system is very much a last resort, and despite its excellent concurrency characteristics the BEAM still lacks basics such as a battle-tested raft implementation.

[1]https://github.com/samsondav/rihanna [2]https://github.com/koudelka/honeydew

> and despite its excellent concurrency characteristics the BEAM still lacks basics such as a battle-tested raft implementation.

A couple of years ago the RabbitMQ team has published a raft library[1] which they use in their implementation of persistent queues. It has a flexible API and implementing your own state machines is quite straightforward, as it follows the OTP gen_* behaviour paradigm.

And by now, I'd say it's pretty well battle-tested.

[1] https://github.com/rabbitmq/ra

I didn’t know about that - thank you for pointing me at it!

Reply on news.ycombinator.com