Hacker News Reader

Comment by technotarek | Hacker News Reader

I'm curious, how would the HN community tackle optimizing (and/or re- optimizing) ~200k images sitting in an S3 bucket?

I would think about using Cloudinary to process these on demand as needed if your usage can support this

Cloudinary Fetch pulls images from your S3, caches them, and serves the size and format you request all without touching the original files

Cloudinary is nice, but it gets expensive really fast on a high traffic site that's image heavy and bandwidth optimized. For example, a single image might have 5-6 versions on top of the original -- low quality thumbnail, high quality thumbnail, default, default 2x retina, default 3x retina. Then activate their webp service and perhaps offer another image aspect ratio. Each is a conversion that counts against your quota. Now you've got 15+ (con)versions and you're shelling out hundreds of dollars a month.

Cloudinary, Imgix, Uploadcare (which I' working for) and others save you money because you don't have to develop and maintain these moving parts. You have to constantly check what's happening with browsers, what formats/encoders are available and which are the best etc.

It's classic build vs. buy dilemma. In majority of cases it's much more cost effective to buy.

BTW, Uploadcare doesn't charge for file processing at all, only for CDN traffic. So you can create as much image variants as you need.

One option is to use https://www.gumlet.com It image processing servers located across the world and ensures very low latency image processing.

Edit: Was thinking more of what self-implemented workflows/precessing tools one might consider (node packages etc) as opposed to paid SAAS options.

Look at AWS batch - https://aws.amazon.com/batch/

Reply on news.ycombinator.com