Amazon bares its computers
bits.blogs.nytimes.com
bits.blogs.nytimes.com
If this is so, how is it that a startup like Digital Ocean that's only 2 years old find a way to provide $5/month plan that's running on SSD while m1.small on EC2 which is slower than DO costing me around $40/month? At this rate, I should probably buy a server myself and run it at home.
I like AWS and its ecosystem, but I am tired of Amazon continuing to provide shitty service for a ridiculous cost.
I think that a lot of the start ups around silicon roundabout woudl be a bit lost if they had to pivot and run their own kit.
the CTO says the VC says we have to switch to self hosting we have 4 racks of dual core 1u 's arriving - anyone know how to put them together :-)
They are one of the few vendors we use that regularly passes on savings to us.
I would expect their costs are based on the mixture of systems they have in place and for it to go down somewhat similar to the market cost for the component, but with a somewhat large lag time as older systems are phased out and newer ones phased in.
If you don't need these additional services, AWS is indeed a bad value. If you use a good chunk of them like we do, it's money well spent, though it won't fit neatly into graphs like the previously mentioned one.
There are a ton of simple VPS providers that would probably get you closer to the baseline for computing costs, but to be fair, those providers are a completely different beast.
Sure most people use AWS as a glorified VPS provider, and for that it is rather expensive. The fact that DO is only 2 years old actually scares me more since they probably haven't faced as many unique challenges as a company who has been in the business for a longer period of time has.
EC2 is a boon for those that need to add and remove capacity quickly. This is the appeal of EC2. It's not going to blow your socks off as far as performance per dollar. It is going to blow you away with its flexibility. If you don't need to scale up AND down quickly and regularly, EC2 is probably not a good fit for you. With EC2, you are paying for that flexibility.
Netflix is one of the penultimate examples of "doing EC2 correctly". Their traffic is extremely bursty. They are constantly automatically scaling up/down, and they do this and so many other things without human intervention.
If EC2 customers realised this, they'd lose 90%+ of their customer base.
If RDS/ElastiCache were cheaper, I think it'd be a lot easier to make the case for EC2 for startups and smaller companies that don't need the auto-scaling features. Even as a smaller business, it's a lot better for us to spend a little more on infrastructure in order to avoid hiring additional people to keep us running.
http://www.nytimes.com/2013/10/25/technology/amazons-revenue...
Analysts estimate Amazon Web Services will have $3.8 billion in revenue in 2013, with a 100% gross margin. It's big enough that it's raising the gross margin of the whole company.
I'm not suggesting the business is not profitable, by all accounts it is, but to understand how it's benefiting the company as a whole, you need to look at the operating line.
If gross margin = 100%, then by definition COGS = 0. Thus, gross margin is a meaningless number in this case.
Everyone thinks cloud is cheaper, but it's more about flexibility and avoiding capex, but at a certain scale, it makes sense to host it yourself - or pay someone to build it out for you.
I'm working in this cloud -> private-cloud migration space currently if anyone is interested in discussing.
My advice to others is to be wary about generalized figures like "20-30% of cloud cost" from anyone with a vested interest in getting you to switch. Generalizations are dangerous when every usage case is different.
Also keep in mind that there is a lot of room for middle ground (even with AWS). For some (but not all), going hybrid can be the best of both worlds.
I have never seen a case where AWS makes financial sense for the base load, or for anyone without that severely bursty traffic, though. For batch jobs or handling peaks of less than 6-8 hours, sure. But fewer people have peaks that are economical to handle that way than they think. And for those who do - e.g. retailers that needs to handle the christmas rush, depending on a system where servers can in theory become unavailable at any time is a high risk game.
Most of the time you can get to 30% of AWS with rented managed servers. Racking your own kit can get substantially below that.
Going hybrid is interesting, though, as it actually makes the case for actually using AWS much even weaker for most people:
Whereas previously you'd need your managed servers to be able to handle sufficient peaks to give you time to spin up more servers, once you have support for spinning up cloud instances for peaks, you can often aim for near 100% utilisation during normal days peaks, which magnifies the price difference substantially.
Even for most e-commerce sites, unless your site has a very localised audience, peaks are rarely high enough or sharp enough to make AWS economical if you have a system that's prepped for a hybrid situation - you can let things get very close to the wire, and start spinning up VMs, but you very rarely will find situations where it's worth doing for more than a few days before it'll be apparent it'll be cheaper to spin up a managed machine billed monthly, and/or putting in an order for more kit to put in your own rack.
> Not having lots of idle capacity that still has to be managed and paid for,
You are still paying for it with AWS with every single hour of VM's you pay for - it's one of the reasons AWS is so expensive: You have to fund all the spare capacity they need to have on standby to prevent running out of hardware during peaks.
> not having to beef up the Ops/IT staff,
This attitude scares me. You have to be prepared to deal with outages of pretty much every kind with AWS too. If you run managed servers, there's pretty much no difference, other than that you have to account for whatever the hosts turnaround time in bringing up new servers is (which one of those places where a hybrid setup might help you - especially if you're using a host like e.g. Softlayer or Iweb that offers the full range from managed servers to cloud servers in the same data centres).
The one thing you don't have to deal with is if you use a colo and it saves you people to cable up and rack servers and install the OS. But you can hire people to do that too.
> not having to own your own/manage/maintain hardware, etc.
... but you don't need to do this to beat AWS hands down, either: If you rent managed servers on a month by month basis, with no contracted minimum term, you can still easily come out vastly cheaper for the base load and most peaks.
> My advice to others is to be wary about generalized figures like "20-30% of cloud cost" from anyone with a vested interest in getting you to switch. Generalizations are dangerous when every usage case is different.
Sure. It's sound advice to do your own maths. But every time I have, for a variety of different sites, AWS has come out ludicrously expensive for anything but batch jobs and very short, sharp peaks.
AWS has turned infrastructure into an api and software which is just simply fantastic, but you pay for that sort of flexibility.
There's nothing wrong with paying for the flexibility, though. We gladly do it because it's a great fit for our traffic patterns and our staffing level.
Elastic scaling will save you money on AWS, but you generally can't scale everything (databases, caches, storage are harder to elastically scale up and down). And you need to pay at peak for whatever your peak is and deal with additional complexity of elastic scaling to begin with. No doubt you're paying smart people already to figure this out - it doesn't come for free.
In any account - if you take a largish box you might rack as an example (Dual-socket Sandy Bridge Xeon, ~256GB RAM, SSD, approximately a cr1.8xlarge), that box costs about $30k annually on AWS. You can buy it for around $8k. Over two years with hosting costs, you're looking at around $10k vs $60k roughly. Numbers work out similarly for other instance types although you would need some scale - perhaps a dozen of these systems - for this to make sense for your application.
Netflix I think is a special case - video delivery which they almost certainly don't do from AWS is their biggest cost - api/web traffic would be dwarfed in comparison, so who cares if it's expensive on AWS in that case? They probably enjoy the flexibility of AWS more than the little bit of money they might save by doing it themselves. Many other largish companies have gone the private cloud route; Zynga comes to mind: http://www.informationweek.com/mobile/inside-zyngas-big-move...). It's not the right fit for everybody, but it's a consideration.
Hybrid is interesting too -- provision your own hardware for the big iron - and burst into cloud for the rest of it.
All the hype of "cloud" makes people think that it's the only and/or best way to do it - certainly not the case for every sort of business.
The important thing to keep in mind with a DIY situation is that there are additional costs besides hardware/rack/space, and a lot of things that you may have a hard time putting a dollar value on (flexibility). It's not accurate to just throw out the cost for a Sandy Bridge Xeon + rack space without factoring in the other stuff (upgrades, maintenance, if you need to add an employee, the lessened mobility, the increased responsibility, etc).
At our current scale, if we went the DIY route, we'd need to hire someone else to take over more of the Ops situation for us. An additional employee would cost a lot more than paying AWS a bit more (salary, management overhead, good benefits, communication). We'd also have more "stuff" to worry about, whereas right now our small team spends 98% of our time working on our product, marketing, and sales. This is entirely anecdotal and specific to our situation, but I mention it as an example.
> All the hype of "cloud" makes people think that it's the only and/or best way to do it - certainly not the case for every sort of business.
I personally can't stand the "cloud" buzz word, because it doesn't adequately describe anything. I am no "cloud" zealot, but on the same token I think some bare-metal-always-wins advocates don't really "get" what AWS is doing or when it's appropriate to use them (even if they appear more expensive at face value). As a developer, I've seen the kind of crazy neat things we can do with the flexibility and the shifting of responsibilities to AWS. For us, AWS is a no-brainer, though we do wish we had a more affordable route to going hybrid than DirectConnect or their dedicated machine offerings. But like I said, I wouldn't dare make a generalized assertion of AWS' value to others based on our situation. Every company/staff/application is different.
So, the question becomes, "How much do I spend on AWS per year?" and something like "If I could hire one person to save me $500k+/year, would I do that?".
If you're a small business with an uncertain future, and smallish AWS spend (<$50k/month) -- sure, stay with AWS until you get into the situation I outlined above.
For example, one of my projects requires 1TB of drive space. At DO, I have no choice but to go to the $960/mo 1 TB plan. But I don't need the SSD speed, 96 GB of RAM, 24 cores, or the 10 TB of monthly transfer. But with EC2, I can just use EBS @ $100/mo/TB and pair it with the right-size EC2 instance for the jobs at hand. As an aside, the real question for me is to decide whether I want to invest in a proper dedicated server where I can get good performance and online storage at a lower cost.
DO has done a fantastic job at finding a spot in the VPS market, which is why I'm one of their customers. It doesn't mean Amazon is doing it wrong.
Thrown in a some consistent hashing and I can scale up/down as needed. Fatcache is completely kick ass.
If all you're after is the storage, that is very expensive unless you keep moving your storage tiers up and down very rapidly. E.g. Hetzner has servers at 49 euro (ca. 66 USD) per month for 2TB.
"Mr. Hamilton later said privately that Amazon had developed original statistical methods to limit damage from catastrophic failures."
I don't think it's what James is referring to (as it's just the tip of the iceberg), but at Re:Invent we also released Infima - a library for service-level fault isolation and fault tolerance using Route 53 for endpoint discovery;
https://github.com/awslabs/route53-infima
The techniques in Infima give some insight into how we think about multi-tenancy and availability, as well as provide a framework for building other services on top of EC2, Route 53 and so on.
So yes, these guys all do lots of tough engineering work, but it's not in isolation and not without assistance from industry veterans whose careers it has been to ensure properly functioning data centers and networks.
Google designs its own ICs now?
They're also the world's largest computer manufacturer (for internal data center hardware).
Software people tend to think chips and hardware design are magic skills known by only three revered people in the world. It's just another track of school you could have chosen. ("Google writes its own software now?")
My god. I used to think I was a bit of a rock star, with (rails) web sites exceeding 100 requests/second on my resume. 1.5 million requests per second ... I cannot even really imagine that.
1.5M relational queries on the other hand--that would be impressive.
For a laughably trivial site, a single host could do a lot of requests per second without too much effort. For a normal Rails app running on a handful of hosts, 100 requests per second is nothing to scoff at, especially if you're on your own.
OT, but for truly impressive single machine performance, check out the LMAX FX trading platform, which can handle 6 million orders per second on a single core on commodity hardware.....in Java. [1]
[1] http://www.arnnet.com.au/article/442144/equinix_expands_its_...
How much storage does one Internet citizen occupy, anyway?
That number chases toward infinite. Its growth may slow down, as we reach the limit of how many 'actions' a person can reasonable take that generates content, but it'll never stop growing just on the basis of the unlimited desire for higher quality experiences (mixed with a general moore's law ride that makes it all financially reasonable).
And just when you think storage demand is going to slow down, we'll jump to some kind of virtual reality life recording, that makes it possible for others to walk around in your daily life as though they're actually there, and storage demand will explode another magnitude.
You can remove Facebook and Yahoo from that list though. Their operations are funded by their core businesses and they don't compete in the AWS-style cloud segment. That is, it doesn't make sense to discuss Facebook in regard to whether it can win or make money, compared to AWS or Azure.
So is there room for Google, AWS, Azure, Rackspace, IBM etc? Yep. It's a future hundred billion dollar plus business (AWS will probably be at $10b+ in a few years). No reason there can't be three or four big players in such a large market.
Amazon's valuation is almost purely an emotional basis, as all future returns are being pulled forward to the extent that they're overvalued and the calculations about their future earnings expectations are correct. Is Amazon worth $50 billion? $100b? $168b? That's an emotional judgment today, as they have no profits and little history of profitability, and if you want to speculate on tomorrow... well if their valuation keeps bubbling up, they'll soon catch Walmart, which has $17 billion in annual profit (meaning it may only take... 20 years or so for Amazon to fully realize that market value via profitability). Amazon is going to be an excellent short at the end of this stock market bubble, they'll lose half their value on the down swing (my opinion).
However keep in mind, this is not a company that has always been loved by Wall Street. For most of its life, it has been hated by Wall Street. If you run an average price from early 2003 through the end of 2008, the stock was essentially flat lined for five years. And obviously it was viewed with near universal disgust after the dotcom bubble imploded, and the stock cratered to the $5's after 9/11. Amazon.Toast and all that.
http://en.wikipedia.org/wiki/List_of_largest_companies_by_re...
Bezos is definitely going for growth.
What are the side effects of this in a few years, when all of the high-end stuff has been designed 4 different ways by 4 different companies?
What happens when Dell and HP's server businesses are shells of their former selves? When Cisco is but a memory as their biggest customers have began designing solutions in-house?
You'll have to go with the cloud. You just won't be able to get the best of the best equipment without doing so.
being a grad student myself, I'm wondering if we'll ever see any specific technical aspects on infrastructures that could drive systems research and provide real problems to be attacked...
I suspect if you had access you would look at a whole bunch of evolutionary improvements and probably trace them back to which papers you've seen it sort of mentioned in.
Probably not as much custom silicon as the article implies either, simply because on the scale of things Amazon is not a semiconductor company and Intel pretty unambiguously own the cutting edge there.