Hacker News Reader

>you need to put that data somewhere else and pass a pointer to it in the actual SQS message

I was under the impression that's the industry standard - you drop the payload in some redis-like storage and pass keys in messages.

View on HN

On AWS you would typically use S3 but yes this is definitely the accepted standard: nobody puts big blobs in message queues because they solve a different problem and are (typically) not designed to handle that.

You will definitely not use S3 in a pipeline if latency is an issue.

"In cases where latency is of primary concern, don't use SQS"

Reply on news.ycombinator.com