Hacker News Reader

Look for job scheduling software plus the name of your programming language of choice. Or background processing. https://sidekiq.org/ is one for Ruby for example, https://aws.amazon.com/sqs/ one that runs in the cloud. Those pages should give you more words to search for as "job scheduling" and "queue" gives too many non-software related results.

View on HN

Amazon SQS is not an ideal choice because it is not designed to be a “forever” queue. Messages will expire in two weeks.

He doesn't seem to need a forever queue. Processing might take a while, but he just needs to get it processed once (and exactly once) and move on. Yes, if he wants to do event sourcing, then yes SQS would not suffice. But I don't think he needs that. But SQS seems to be a great choice.

Yeah we are already using SQS for a similar system that's currently in production. We don't need a forever queue as part of the goal is to keep the queue as empty as possible at all times. What is really important is making sure that only one job (with a distinct set of params) can be scheduled at one time and run by only worker at one time.

Reply on news.ycombinator.com