There is a bug when using the edge-triggered fd from a zmq socket. I am not sure if it's fixed upstream yet or not. See here for an ugly workaround: https://github.com/dotcloud/zerorpc-python/blob/master/zeror...
Also, this looks to be a fix in recv(), I am having issues in send() hanging randomly blocking the entire process. I ended up using a with timeout block around it so if send blocked it would eventually get back to me...
sent = "WAITING"
with gevent.Timeout(0.5, False):
sent = self.socket.send_multipart(tosend)
if sent is "WAITING":
print "__incoming_consumer: Timeout fired"
# We are going to try again
with gevent.Timeout(2, False):
sent = self.socket.send_multipart(tosend)
if sent is "WAITING":
print "__incoming_consumer: Timeout 2 fired"
continue
gevent.hub.sleep(0) # Yield to other gevent's, we can be fast and never let up ...
This fixed it for a little while, but even then every so often it would hang, and it was causing us to have to restart our frontend processes (accepts incoming connections for processing) so we decided it was worth the time and effort to re-write it in C++ with libev as our event handling mechanism. So far we have put it under more load but have not had any lockups or failures.