aws lambda for me. Just send that data out and get it later when it’s ready. Run a simple python function to do your processing on the aws side.
So far I’m pretty happy with it. I minimize the need for maintaining a redis + worker etc
So far I’m pretty happy with it. I minimize the need for maintaining a redis + worker etc
I'm using Lambda to kick off an EC2 worker running a Docker image with AWS Fargate. The EC2 instance only runs for the duration of the job (which lasts from 5 to 20 minutes)