Fast, Ordered Unixy Queues over NNCP and Syncthing with Filespooler

changelog.complete.org

34 pointspabs32 comments

I didn't take a deep look at this, but the general idea of setting up a worker queue as a filesystem structure & synchronizing via sync-thing is pretty cool.

Only question is, would this support multiple-workers? Would be neat to be able to distribute work a to a cluster this way.

I think Syncthing is an under-appreciated tool! I wrote more about it at https://changelog.complete.org/archives/10219-a-simple-delay...

https://www.complete.org/parallel-processing-of-filespooler-... discusses parallel processing in Filespooler. This isn't a design goal and may not be the best fit, but when everything's just a file, some tricks are possible here.

Another interesting use case is one-to-many and many-to-many; https://www.complete.org/filespooler/ describes those.

Reply on news.ycombinator.com