Hacker News Reader

Can you consider Dask integration? http://distributed.readthedocs.io/en/latest/queues.html https://github.com/dask/dask

It can handle parallel and distributed parts for you.

I just read about dask earlier today, very neat project! riko already handles parallel processing [1] but adding distributed processing sounds tempting. TBH though, distribution isn't high on the priority list. But I'll be happy to accept a PR if you are so inclined :)

[1] https://github.com/nerevu/riko#parallel-processing

Reply on news.ycombinator.com