Hacker News Reader

Can Celery help handle state internal to the workers?

What about using Pkykka alongside Pyro such that each Pyro remote object is actually a Pykka actor? Such that Pyro allows splitting workers across separate Python OS processes and the "messaging" while Pykka handles the internal state.

View on HN

Could you give an example of state internal to the worker?

Similar to how one can pass around state inside an OTP GenServer in Erlang.

In Python, it could be a class managing a session type of object, like a TCP socket connection, or managing some piece of hardware that is doing something independently of the other parts of the system, or a database writer, etc.

Oh sure, if I'm understanding you correctly. It's been a few years since I've wrenched on such code, but IIRC, you can pass any seralizable object you want to the function being executed. We used to pass around an in-memory class object, no problem. The tricky part was control flow, but that wasn't celery's problem.

Reply on news.ycombinator.com