Hacker News Reader

Comment by funstuff007 | Hacker News Reader

Not be that guy, but we never send an object to S3 without compressing it on our end first.

We never send anything unencrypted to S3. I have no clue why someone would not want to encrypt their data before putting it in the cloud…

For people also using other AWS services (in particular compute ones like EC2) this doesn't do anything meaningful.

We use other aws services. We decrypt the files on those services.

Because there is already server side encryption. Which I guess means data on disk is always encrypted not sure about in-memory.

We also use server side encryption. But honestly encrypting the files before putting on s3 has given us 0 grief and gives us a feeling even if we somehow are “hacked” the files are useless to the hackers.

Where is your decryption key stored?

Post-it as a Service

We have a one and done rest api that will only send the key once after being staged as part of our pipeline. So that’s the delivery mechanism. Key is on an internal server that isn’t on the public internet except the one and done. Super simple.

Would you encrypt files that are supposed to be publicly readable and accessible?

I get hashing it and providing hashes, but encrypting public files seems excessive.

There's a very common data/ETL pattern wherein raw (unencrypted) data is stored into S3 at the very beginning of any pipeline. Adding encryption adds a layer for failure, which can grind your pipeline to a halt.

I've seen a pattern of: drop raw data into an S3 bucket that has a very restrictive policy with a long retention policy. Then, process that data asynchronously (encrypt, transform, filter, etc) and drop it into a different bucket/area that is accessed by other consumers.

Then, if any part of your ETL fails (encryption included), you can fix your bug and reprocess from your raw data without writers seeing any impact.

Reply on news.ycombinator.com