I guess this should be clarified more in the README - this uses zmq (and potentially other brokers) as the queue, and uses the database for tracking results, saving recurring tasks, etc
Where's the actual queue though? zmq isn't a message queue, it's more akin to a protocol. e.g. if I have N workers and M jobs, and M>N, where are all the >N indexed jobs kept while the others are being processed?
Like it appears you're using PUSH/PULL pattern, which means the jobs queue up in the publisher's memory (on a per PULL-peer basis..). So if the publisher crashes, those jobs are lost. If the number of jobs queued exceeds the high-water mark, those additional pushes are either going to discard messages or block. If it blocks, it looks like your code is going to have lots of other issues... If the worker crashes, I believe that queue is also going to be lost.
If I am understanding the other notes correctly:
Neither the worker nor publisher should ever really block (even in cases where M>N, N>M, publisher crashed, etc), mostly due to the async zmq bindings for node. If there are a ton of messages being sent to a single worker this worker will pull down messages as fast as it can and the rest will still be in publisher queue.
If a worker crashes this is happening separate from the publisher and won't affect the queue stored in the publisher's memory