Hacker News Reader

Comment by ridruejo | Hacker News Reader

The advantage of AWS is the scale at which it can operate and the rich ecosystem of services (S3, ELB, CloudFormation, RDS, etc) that you can use as building blocks for your app. DigitalOcean, Linode are more like VPS providers and are only suitable for a subset of the use cases AWS covers (or Azure or Google Compute Platform)

View on HN

If you are/have skilled sysadmin staff, there is very little if anything AWS offers that you can't run on your own rented/owned phsyical/virtual servers.

I'm not suggesting that's ideal for all companies, but this idea that somehow it's impossible to run a load balancer or store files without using AWS/Azure/Google 'clouds' is a bit ridiculous.

I don't think replicating S3 is all that easy for any company.

Running an FTP server != an S3 equivalent.

For the vast majority of people, S3 means cheap, mass storage.

For the majority of those people, a simple SFTP/rsync/NFS/whatever endpoint (potentially with replication to provide redundancy) would more than fit the bill, and would actually be simpler to use than the S3 API.

I store about a terabyte on S3. It costs us $25 a month. If I spend more than a few minutes a month administering an alternative setup, I've lost money compared to just paying S3.

That entirely depends on how you use the data. If you store files that are accessed over the general internet, and each file is downloaded just once in the month, you've just quadrupled your monthly cost, thanks to AWS' high data transfer fees.

I've spec'ed out S3 alternatives for clients, and pretty much no matter what criteria they have for redundancy, managed hosting solutions or co-location comes out much cheaper than S3. E.g. typically half or less of the S3 costs for triply redundant servers, on RAID arrays, in separate data-centres with replication (e.g. using OpenStack Swift, or Gluster / Ceph + an API layer for example)

For larger setups you can do it at 1/3 or less of the cost if you're prepared to go the colo route (rather than managed servers) and leasing your own hardware.

If your average bandwidth usage per object is high you can trivially cut costs far beyond that, as the EC2/S3 bandwidth costs are atrocious.

The exception would be if you can't avoid huge amount of object accesses from within EC2 even if you move the storage out of S3.

Are there any resources out there that teach developers weak on scaling/architecture how to manage this themselves? What architectures are the "best practices" of today? What tools are proven? Any books/articles/papers to recommend?

> Are there any resources out there that teach developers weak on scaling/architecture how to manage this themselves?

The same resources that Sysadmins used to learn how to do it?

Or hire a sysadmin part-time.

Good point, it becomes clear that if you are operating at the scale of a large company the large scale and rich ecosystem are important. It was just a bit surprising to me that for smaller scale instances AWS is so much more expensive. It would seem they should be cheaper due to economies of scale.

AWS is definitely aware that their costs can be prohibitive for brand new startups. Check out AWS Activate[1]. If you can demonstrate that you're a legit startup, there's a decent chance you can get free AWS credits, not to mention free support.

[1] http://aws.amazon.com/activate/

Getting the business level support is absolutely necessary if you don't have someone with AWS experience and are doing anything substantially complicated. The free-ish support is, on the whole, pretty abysmal. The availability of solution architect time is not always useful. You ask 3 people and get 3 different answers. I've given some thought to doing consulting around this for equity but haven't figured what that would have to look like. There's a lot of wrong turns one can go down pretty easily. Bottomline, Activate is helpful but not sufficient.

Consider that whenever you are paying for AWS you are also paying for whatever proportional part of the AWS infrastructure is sitting idle at any one time - Amazon still need to cover those costs - it just get amortised over the paid resource usage.

Reply on news.ycombinator.com