Hacker News Reader

Comment by chrisandchris | Hacker News Reader

For my use-cases, this would be a huge benefit:

> Resizing and watermarking images on the fly using caller-specific details, such as the user who requested the object.

Now we need to either do rescale directly after storing or need to use another service (on EC2) to do the resize. With Object Lamda, we could do it on-the-fly as the user calls the image. This is operational a lot easier.

View on HN

Even without this feature, you could stand up a lambda and do the resize on the fly.

Advantage here is that clients can just continue to use their existing S3 interface yet your lambda is still invoked.

Does this get triggered over the S3-over-HTTPS (the one used for websites) served objects? (instead of S3 APIs)

The benefit of S3 Object Lambda is that your function is adapted to the S3 API. If you already have an application integrated with S3, then you can use Object Lambda to provide some new behavior to the data without changing the application.

How are parameters passed? That is how does Lambda get its resize parameters from just an S3 GET? Are they query params?

In the example use case for that they suggested baking the size into the filename, like image_500_240.jpg

You might want to build some rate limiting into your lambda as well!

At the moment this is currently handled with lambda @ edge with Cloudfront. The benefit with Cloudfront over S3 lambda is that the image is automatically cached after the resize is computed.

You could cache the image with s3 lambda too, no?

Edit: I guess you meant that cloudfront caches it for you automatically.

Exactly, yup. Whatever it returns is automatically cached at the key.

Your example is way better than what AWS has on their page (marketing, analytics) which made me think immediately: you shouldn't use a file store for that but a database.

S3 is basically a database. You can store everything there and interact with it via SQL with Athena, for example. I could see utilizing all of Amazon's "infinitely scalable" services to build very simple and powerful software really easily.

I think Athena has to get a lot less expensive for that to become true.

For small numbers of large queries, Athena has been an incredible technology stack .. I pay pennies a month to run my analytics jobs on it (aggregates across a few hundred million rows, joins with tens of millions of rows into hundreds of thousands of rows).

I start my data as .csv.gz but the first step is a CTAS to extract columns and convert to compressed parquet. This step basically costs the most but gives a 10x data size reduction to downstream steps.

Athena does not work at all if you perform large numbers of small indexed read queries, definitely use a traditional database for that.

Yes, it's great if you setup the data structures properly, we pay a few bucks a month to run a lot of infra on it to query telemetry data over a few hours to a few days at a time. I think an equivalent cost in just a database server instance to run the same load would be at least 25 to 50 a month. You're not really locked into athena, either, it just takes standard sql, so you could easily transition to a dedicated box later.

I copied an example from the AWS blog post and adapted to our use-case.

I did never work with it, but there is Athena that allows to query S3 in something like SQL. I think there _might_ be some cases were S3 is a useful database, as one can use pre-signed URLs to let any client write to the database (S3). That‘s something I‘m not sure how to set up with a classic database and without any API in between.

IMHO, I think it‘s all about the same functionality with a simpler architecture.

Reply on news.ycombinator.com