Hacker News Reader

Same issues updating a file - you’d update on s3, then trigger processing and even though it was 50ms later - you’d get OLD version. I didn’t know gcp didn’t have this issue - would have been a big selling point

That sounds _very_ wrong, what type of throughput are we talking about here?

The story wasn't wrong, until today. That's why people are celebrating.

Up to hundreds of images per second.

Okay that makes sense, when we have very low throughput it looks like consistency wasn't an issue but at that rate (especially if it occurs in bursts) I can see how the prior S3 implementation could fail to scale fast enough

Out of curiosity, did you have retries on the GETs?

No, we didn't try retrying (well actually if the job failed because of a 404 it would retry the whole job based on exponential backoff). When we encountered the error we simply added the delay as it was an easy config change to our SQS queues and doesn't meaningfully affect our use case.