Hacker News Reader

Comment by erikbye | Hacker News Reader

What does everyone use S3 for?

I'm genuinely curious. As my experiments with it have left me disappointed with its performance, I'm just not sure what I could use it for. Store massive amounts of data that is infrequently accessed? Well, unfortunately the upload speed I got to the standard rating one was so abysmal it would take too much time to move the data there; and then I suspect the inverse would be pretty bad as well.

View on HN

S3 uploads can be very dependent on what the network looks like between your systems and the endpoint. We introduced S3 Transfer Acceleration to help address this. This uses our edge network as your endpoint, then sending your upload across our backbone rather than traversing the commercial internet. It comes with a small fee, but the fee only gets applied if it improves on what the time to transfer would be otherwise.

https://aws.amazon.com/blogs/aws/aws-storage-update-amazon-s...

If you have lots of data that needs to be uploads (TB/PB worth), then I'd take a look at AWS Snowball. https://aws.amazon.com/snowball/

Also, if you're using the AWS CLI to upload, make sure multi-part upload is enabled.

Why use S3? It scales without any user interaction, is highly available (yes, I cringe saying that, but this is the first, and hopefully only time this has occurred! =D ) and extremely easy to access; it's as simple as an HTTP GET. Being able to address objects directly and not have to worry about managing file systems simplifies a lot.

1 scenario: if you run a website that has a lot of static content (multiple GB of images, css, js, etc) and you dont want your http server to be responsible for serving that content then you give it all to s3 and let them serve it for you.

What about the performance of serving it? Sounds like I would need to cache it myself, anyway.

Performance out of S3 is generally really good. However, if you're looking to say, serve up a global website and your content is in a single S3 region, then you can leverage CloudFront CDN to serve up those objects. CloudFront integrates seamlessly with S3, and you don't pay transfer charges between CloudFront and S3.

It's a very easy way to cheaply store static data to be served over HTTP. There are many many many many many many people who want to do that.

Database backups, puppet modules, a couple low traffic static sites, assets for one of our projects.

Database backups: with the upload speeds I've seen, completing a backup of a database with hundreds of GB would take a really long time. I don't want that extra load and keeping that connection open forever.

S3 endpoints in your VPC?

We are storing database backups and static assets (e.g. images in mails) on S3.

Reply on news.ycombinator.com