Hacker News Reader

Comment by johansch | Hacker News Reader

Exactly. I would think very, very, very hard about building something on top of any AWS technologies for which there are no good open substitutes of.

View on HN

It's a trade-off.

As far as I am aware, there is nothing out there that can rival the cost and functionality of S3. I'm not going to avoid using it simply because there is a lack of open alternative - it's that much better that I will use it now and redevelop to use different technology if and when I have to.

After all, most of these services all have pre-made client libraries. Rewriting code to use a different service isn't likely to be world-shattering.

I think your own cluster of for-rent machines would outperform the price performance of S3 wildly. Using e.g. Riak or something similar.

(I haven't done this particular pricing calculation, but everything else in AWS seems awfully overpriced, so why not S3?)

But then I have to maintain a Riak cluster.

For all means, if you have LOTS more money than time, you should pay someone else to do this. Outside of certain sheltered silicon valley startups, this is often not the case though. :)

You know, I'm pretty sure that almost everyone is better off spending a few cents per gigabyte per month on S3 for inert file storage, a functionality that's performance-inelastic except at the far end of the curve, than maintaining a Riak cluster.

Source: I've maintained a Riak cluster.

There are lots of AWS services I avoid (I won't use SQS, I won't use Dynamo except during prototyping, etc.), but S3 is so utterly portable across any cloud you go to (and latency is already not something you are worrying about if you're using S3) that it's laughable to suggest that you're going to roll your own with anywhere near the reliability at anywhere near the cost.

We rolled our own proprietary version of what became known as S3 and scaled it up to several thousand nodes. At any given time, less than half a FTE spent any time on this, and I think we've had roughly equal (probably better) availability than that of S3, at a fraction of the cost.

Assuming you pay your engineers a paltry $50k/yr, half of a FTE is $25k/yr. Quick calculation based on the public S3 pricing says that's 847TB of data you could be storing just with their salary. That doesn't include all the actual costs associated with running thousands of nodes - acquisition, hardware replacements, utilities and other operating costs, space, etc.

(To be honest, with that many nodes I have a hard time believing that you're not spending half a FTE just replacing dead drives.)

Not to mention the additional costs if you're actually interested in the same level of durability (target eleven 9s (99.999999999%), but last time I talked to an engineer they're still at 100% since leaving beta). Multiple DCs, transit between them...

I'm sure there's a scale at which it's worth it, but it's pretty far up there and for almost everyone S3 is a cheaper option.

I spent zero FTEs on S3 and average around twenty-five bucks, or under a third of an FTE's hourly amortized rate, per terabyte.

You might not be in "almost everyone", but almost everyone else is.

Yeah, we use S3 at the company I work for a little under a terabyte and we pay less than $25 a month.

I think you're overestimating how expensive S3 is. You'd have to be storing a lot of data before running your own infrastructure becomes more affordable (especially factoring in employee costs)

Not to say it never happens, but 90% of people are better off just using S3.

Why wouldn't you take advantage of everything your platform gives you? This goes for your OS as well as your infrastructure.

As far as I know, the biggest reason not to rely unconditionally on AWS-specific offerings is the same reason to avoid vendor lock-in elsewhere: They've got you over a barrel when it comes time to re-negotiate pricing. Amazon has a pretty wonderful history of price cuts, but when Google comes in with a better deal (or you want to hedge your bets with a hybrid cloud-type thing), it's waaay harder to find analogues of things like Kinesis and SES and Elastic Beanstalk.

There are definitely some PaaS-type offerings they have that aren't as scary (RDS/MySQL, for example, and ElastiCache/Memcached, etc.), but if you're trying to bounce systems between clouds, or deploy the exact same configuration everywhere you're running, the only real answer is to rely on compute resources (basically EC2) and build your own stuff. All the clouds have Linux servers.

I agree with your general point, but switching between Kinesis and Kafka isn't too crazy. SES can be replaced by Mailgun, Postmark, so on and so forth.

The AWS services I use without hesitation are EC2, Route 53, S3, SNS (to mobile devices, not to my applications), and SES. RDS and ElastiCache can be okay for early, pre-scaling situations, and Dynamo is okay for proving out a problem before cranking out a Cassandra cluster (but I wouldn't want to scale on Dynamo, gets real pricey real fast).

I will also heavily recommend Redshift, because it's fantastic for a lot of different offline workflows and it's a lot nicer than maintaining your own. It works well without being tied to AWS for your actual infrastructure.

Because you want to avoid locking your future with one particular service supplier. Particularly one that charges so much (for e.g. bandwidth) as AWS.

Still cheaper than what google charges for bandwidth.

Might be. It's still 10-20x the market price.

Reply on news.ycombinator.com