Rackspace cloud beats Amazon EC2, by a lot | Hacker News Reader

Rackspace cloud beats Amazon EC2, by a lot

scripting.com

203 pointsdavewiner76 comments

I've conducted extensive benchmarking of rackspace cloud, ec2 and about 38 other iaas providers. Rackspace cloud is definitely not faster than ec2 by a long shot. Rackspace cloud utilizes homogonous infrastructure, AMD 2374 to be exact. All instance sizes are burstable, so you typically get about the same CPU resources on a 1GB instance as you do on a 16GB instance according to our benchmark results. Ec2 on the other hand scales CPU much better all the way to dedicated dual quad X5570s with the cc1.4xlarge. Both this and the rackspace sponsored bitsource study compared an ec2 m1.small. Any study that does this should be immediately discounted as that is about the worst possible performing ec2 instance size. Adrian cockroft from Netflix refers to these types of studies as benchmarketing. They do not accurately depict the performance capabilities of ec2.

http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma...

I think EC2 and Rackspace Cloud serve two different groups. EC2 is the only provider on which I've actually been able to boot 50 nodes, have them come up in a few minutes, use them for an hour, and kill them all off. That sort of thing would be a giant pain on Rackspace Cloud, since they e-mail you the root password when you boot an instance. Also, Rackspace Cloud accounts are limited to 50GB of RAM usage unless you contact them to increase the cap. (Rackspace only mentions this in their API docs: http://docs.rackspacecloud.com/servers/api/v1.0/cs-devguide-... See section 3.8.2: Absolute Limits.)

Still, most small and medium-sized companies would do best to go with Rackspace, Linode, or something similar. You'll get better support from them and it's not often that a 10 person company needs a ton of servers for a short period of time. Even then, you could use both: short-lived instances on EC2 and stable, well-supported, long-lived stuff on Rackspace.

It's actually quite easy to spin up multiple servers from a pre-existing Rackspace image via the API. The initial POST to create the server returns the password, which your script could either capture, or you could send a PUT command to the /servers/id URI to update the root pass to be whatever you want it to be.

I didn't know that. Thanks for the correction.

That's not my only reason for preferring EC2 for lots of short-lived servers. I left out some anecdotes.

Four months ago, one of my coworkers booted 12 cloud servers in DFW. I later discovered that 3 of them were on the same physical hardware. Two months ago, about 55 of 60 servers actually came up in ORD. Others were inaccesible or hung. Not even hard reboots helped. We had to kill them and start new ones.

I've had a total of 2 EC2 instances die on me. I admit my usage of the two providers is quite different. The stuff on EC2 is shorter-lived. But I'm pretty sure that high turnover on other providers would cause a lot more grief.

I've seen literally dozens of unresponsive and defective EC2 instances over the last year. This was spawning 100s of medium and large instances per day, with an average instance lifetime of around 3 hours.

From what I've read during my usage of AWS over the last couple of years, this is more the norm than the exception.

I think we agree then. Dozens out of 36,000+ instances is a very low failure rate. I haven't had nearly that volume so I've only experienced 2 failures on EC2.

In addition the Rackspace API supports injecting a file into the API post request which will be placed into the system as it is initialized. The intent of this is to inject some sort of individual identifier to a server, which could be in the form of an ID number that is read by a boot script in your gold image, but other useful scenarios that would work would be to inject the root password you want or to disable root login entirely and inject an ssh public key file for a non-root account.

We do this (the ssh key). It's dead easy.

For what it's worth, I ran a trivially complex system on Rackspace cloud about a year ago and it was a total clusterfuck. My machines were rebooted all the time and I would receive e-mails saying they were rebooted by Rackspace because of infrastructure issues or maintenence. I'd say this happened once every 2-3 weeks and I was only running 3 servers. This was a hobby project so to have to drop what I was doing every 2-3 weeks to reinitialize a server was a huge pain. I eventually shut it all down and switched to Linode just so I didn't have to worry about them randomly rebooting my machines all the time.

I've ran much larger clusters on EC2 over the last several years (50+ servers) and can count on my fingers the times that machines have been rebooted. And when they have, it's due to a lightning strike or a AWS failure that's reasonably explainable.

Rackspace lets you burst CPU, whereas Amazon EC2 does not. It's not terribly surprising that Rackspace is performing better in that area.

What you need to be careful about is the fact that EC2 and Rackspace Cloud are currently in two different leagues when it comes to controlling your instances. Amazon has a far better control panel (not to mention the API) for exercising granular control of your instances. Hard drive space is dynamically scalable on Amazon whereas it is not on Rackspace (their solution of mounting Rackspace Cloud Files via FUSE is unacceptable).

The monitoring system for EC2 is also far better, and completely non-existent on Rackspace unless you pay them $99/mo out of pocket on top of your hourly usage charges.

All in all, my Rackspace experience left a very bad taste in my mouth when dealing with their support, and it was a culmination of the small and simple things that left me frustrated (the lack of pv_grub support out of the box, etc., etc.) and kept me on EC2 despite the lack of CPU bursting.

Rackspace bought Cloudkick. I would expect the built-in monitoring to improve greatly in the near future.

I've used both fairly extensively now and there are definitely pros and cons to each. I haven't done any solid benchmarking, but RS feels faster. It's also much easier/faster to get up and running whereas AWS has a bit of a barrier to entry. While at times I've been given incorrect info from them it's great to be able to get support staff on the live chat.

I wouldn't necessarily choose it for all applications though. The AWS architecture is far more flexible in general. Elastic IPs are invaluable when it comes to creating a system that can grow over time (seamlessly switch from a single instance to several fronted by a load balancer in a couple of minutes). Being able to take complete snapshots of your system on an hourly basis could well save your business one day. Being able to make a couple of API calls to attach an extra 1 TB drive to your instance? That's worth losing a little horsepower over.

It all depends on what you're after; with Rackspace you probably get a faster machines, but that's at the expense of being able to build a more robust generic solution for your needs.

We're on Rackspace, and getting murdered by a couple of things:

1. Not being able to idle a system, or to restore from a system image (some persistent bug on their side w/r/t setting netmasks on external interfaces, of all things);

2. Not being able to buy disk independently of RAM.

We were moved from DFW to ORD and since then, we haven't seen the random weird outages that had me pulling out my hair. It hasn't been bad enough to make me want to move to EC2/some other hosting company, but I do look longingly at, say, spot instances, which would be a perfect tool for some of our problems. I'd love it if FreeBSD worked correctly, but I'd also like a pony, so what the hell.

#1 doesn't sound right to me. You should be able to create Cloud Files persisted backups and use them to launch new instances even after the original parent is gone.

Granted there could be some extenuating circumstance for your specific setup, but I think typically that scenario is supported out of the box.

It's a fairly specific case, yeah. We've automated image construction, but it's still a PITA compared to just spinning up a new instance.

That said, there are always bigger fish to fry.

When Cloud Servers work, they work great.

The pricing and build-out structure is linear when looking at RAM & disk space, so this may or may not fit everyone's requirements. There is no EBS-equivalent, and load balancing has just been introduced formally into the control panel recently. The persistency is something I've taken advantage of, compared to EC2's ephemeral nature (unless one employs EBS).

As for cloud servers going down randomly, Rackspace Roulette can be tough, and the only silver lining is it provides a good incentive to build (or at least, to think about) applications which work around failure.

There is one other lesser known gotcha which is max RAM capacity per account; I think the default is something like 50GB and if you require more (for burst perhaps), you have to get this amount pre-approved. Apparently this is to safeguard against (accidental) abuse of the Cloud Servers API, but it's probably also a good mechanism for capacity planning on their side. At any rate, I've seen/heard the turnaround to be about a couple of business days.

One the flip side, there was a very active thread a while back about how Mixpanel moved away from RS: http://news.ycombinator.com/item?id=1884685

EDIT: One more note about RS -- resizing cloud servers. It's a great ability to have, but it can be slow. It tends to take longer the more data you have (not too surprising). A good practice is to wipe unnecessary log files before resizing, as I've been told that each resize action actually causes the cloud server to jump to another physical host. Don't quote me though, I'm just on the Internet :)

I chose the cheapest option on Rackspace, a 1GB 32-bit Windows 2003 server that costs $0.08 per hour, which works out to $59 per month. Significantly less than the $90 a mini-server costs on Amazon.

Is that supposed to be cheap? I used EC2 for some compute tasks a week ago; it was 23 cents per hour for an x86_64 8-core 2.16Ghz i7 system with 8 gigabytes of RAM -- which sounds way more than 3 times as powerful as the system they mention.

Running on "burst CPU" doesn't sound like a very useful strategy when I need to load a few dozen cores for a few days.

Choosing Windows on either Amazon or Rackspace adds quite a bit to the cost, and it rules out using the very cheap instances (micro and what not).

Windows 2003 micro instances have been available for quite some time now on EC2.

Ah, didn't realize that.

For one of our sites, we ran unixbench on EC2, Linode and StormCloud (LiquidWeb)..and I can tell you that EC2 wasn't the best value with respect to price/performance (not by a lot).

I can also tell you that having not picked EC2, I have, more than once, wished for Feature X offered by Amazon. Maybe Rackspace Cloud offers a comparative list of addons/products, so my point might not be too relevant. But, my point is that price/performance shouldn't be the only determining factor.

An in-depth performance comparison published by an independent third-party: http://www.thebitsource.com/featured-posts/rackspace-cloud-s... Rackspace Cloud appears to perform much better than EC2 in the States.

This study was sponsored by Rackspace. It is based entirely on 2 benchmarks - iozone and linux kernel compilation time.... not exactly a thorough study. It also only tests m1 instances and ephemeral storage. Not a great study imo.

better: http://cloudharmony.com/benchmarks

The biggest reason I've stuck with EC2 is that no other provider provides many hundreds of GB of disk space like EC2 does. Even the smallest "tiny" EC2 node can have 1TB of EBS attached to it.

Yes, this is indeed a big deal and (as far as I know) unmatched so far.

EBS has been getting a lot of bad rap recently (most of it deserved), but being able to juggle detachable, snapshot'able 1TB-volumes for $100 a shot has still been a game-changer for many companies.

Yeah - my whole architecture is based on simpleDB....

Honest question: could this just be because no one is on Rackspace?

Doesn't seem likely. VMs impose limits on resource consumption (though Rackspace allows CPU bursting, from what I understand). It's not like the old days where you could get lucky on a sparsely-populated shared server.

Rackspace cloud servers are basically what slicehost offers. They don't have cpu bursting. Instead each slice on the host is guaranteed a minimum amount of cpu and any unused capacity is available for other slices to use.

So if you have four slices on a host and 3 slices only use 10% each of total cpu capacity. Each slice is guaranteed 25% but they aren't using all of their guaranteed cpu. Your slice can then consume the remaining 70% of the cpu on the host. (Minus virtualization overhead)

Unlike bursting, you don't get throttled until other slices need cpu. Your slice can happy keep consuming 70% until the other slices start asking for their share.

They ARE what Slicehost offers. Same company, IIRC.

I know Cloud Servers originally started as a direct copy of slicehost with hourly rates and end of the month billing instead of monthly prepaid billing (prorated per day). A few things have changed since then and I don't know if slicehost and rackspace are still offering the same system any more.

I do know that the vm environments are the same. I'm just not sure about some of the other bits.

If you go by the look & feel of their web interface then it certainly still feels like a bolt-on. I also remember various of their server ips to have PTRs to slicehost names, but that may have changed since I looked (a couple months ago).

Still happens. Mine are all still pointing to slicehost.com.

Prorated hourly, actually. Which is really really nice if I only need to spin up a few clones for a little while.

I thought Cloud Servers were from Rackspace's Mosso project and Slicehost remains mostly independent?

No, Cloud Servers are Slicehost. Cloud Sites are Mosso.

It could be.

There is definitely a different target customer and that matters. Rackspace customers are more likely to be classically architected n-tier web sites/apps and AWS customers are more likely to be fully "cloud architected".

The result of this difference is that a Rackspace cloud physical host is not going to be as heavily taxed resource-wise (at least on average). Add to that the cpu burst by default versus cpu cap by default and cpu is definitely the one area where differences will be widest.

One thing I will say though, and many may disagree with this, but I have yet to see a benchmarking study that I'd trust. Real world operating performance of a server is just a very complex thing and any attempt to distill it down to single parameter comparison is going to compromise some element of that complexity. I can show you comparison studies that show any of the cloud providers to be better than the others, including AWS, Rackspace, Linode, Joyent, etc. So take these things with a grain of salt.

> ny attempt to distill it down to single parameter comparison is going to compromise some element of that complexity

This is extremely true not just of this but of comparison in general. Turning something complicated into a natural number below 100 and then using > and < is pretty limited.

My practical experience does not agree. I've actually started moving off of Rackspace cloud because I find that sometimes it is super slow. No such problems with EC2 in three years of use.

The new Micro instances on EC2 make it a no-brainer.. really.

I more or less understand the reasons, but I do think it unfortunate and weird that the companies offering cloud services price them on some combination of RAM, disk space and bandwidth--CPU performance doesn't really figure, except occasionally in vague terms, and neither does disk access speed.

RAM is important, sure, but there should be some way to quantify the expected CPU performance as well.

Can anyone explain why cloud hosting is so expensive? I can get i7 quad core with 8 Gb RAM for $60/month from dedicated servers providers, but i can't find comparable price with cloud hosting.

Could you share where you get these prices?

Hetzner: http://www.hetzner.de/en/hosting/produkte_rootserver/eq4 (you should be outside of eurozone to get -19% VAT from the price)

Server4you: http://server4you.net/root-server/ (haven't tested them yet)

Thanks.

BTW, server4you have a lot of horrible stories. Hetzner reviews are better, but it's important to understand that this deal hardware is of "desktop quality" and not as reliable as "standard server" one. And it is in Germany.

Brand servers won't get you reliability, only failover and replication will :)

In US servers are a bit more expensive, but you can get similar configuration for around $100/month (e.g. burstnet)

In fact I think so too. But I suppose there are some cases for some people when the brand hardware would be better.

Tangentially, I was just looking at the possibility of hosting a file on Amazon S3, and it's astonishing how expensive it is. They want $0.15/GB for bandwidth.

In contrast, Linode sells bandwidth for $0.10/GB, and that comes with a whole VPS. So if I pay $160/month, I get 16TB of bandwidth on a VPS with 4GiB of RAM and 128GiB of storage (oh, and Linode pools your bandwidth across all nodes). On S3, $160 will buy me a little over 10TB of bandwidth, nevermind storage or anything else.

I understand that these cloud services are the most convenient way to scale, and probably the best way to absorb an unexpected spike in usage. But they seem to charge a roughly 50% premium for every resource, as compared to a reliable VPS provider with great customer service, an API that lets you set up temporary servers, etc.

> So if I pay $160/month, I get 16TB of bandwidth on a VPS with 4GiB of RAM and 128GiB of storage.

In Linode, for $159.95 you get 1.600GB or 1.6TB of bandwidth. Not 16TB.

Linode to S3 is not apples to apples. If you want to compare to S3 you would need multiple linode boxes and a system for redundancy in the face of image corruption or hardware failure.

One of the things you are paying for with S3 is a pretty solid guarantee that your files will A) not get lost and B) always be available.

B) always be available.

That's a relative term, though. While complete outages are rare, there are frequent, prolonged periods of abysmal performance (latency in the >300ms range, throughput approaching <50 MBit/s) and the average performance without CloudFront isn't stellar either.

However, for many use-cases the sheer convenience and low cost of entry trump these issues.

You can't really compare S3 to a VPS. S3 offers redundancy and scalability that nothing running on a single VPS could do.

Yeah, but how many people take full advantage of that? How many would be better served by a beefy Linode instance running lightthpd?

Redundancy is insurance. By the nature of being there it's taken advantage of

http://www.lesslettuce.co.uk/ is running on Rackspace cloud and we've not had any problems whatsoever with it. Got nothing to compare it against though, haven't tried EC2. But this makes me glad we went with RS in the first place!

There's one of these every week. Someone is using either Rackspace or EC2, switches to the other, and talks about how the original is terrible.

It seems pretty split as to who likes which provider. Don't let one blog post sway your opinion too much.

At Cedexis, we compared cloud performance using 15 billion measurements from actual end-users in 220 countries and 23,800 networks between:

- Amazon, Rackspace, Joyent, Google App Engine and Azure and

- How do EC2's East, West, EU & APAC zones compare

http://dl.dropbox.com/u/1898990/76-marty-kagan.pdf

Drop me a note if you are interested in additional information, a similar test for your website -- prakash at cedexis.com

Shameless plug: we recently launched a service to help folks discover and evaluate different cloud computing offerings. In addition to Rackspace & EC2, you might consider Linode, GoGrid, SoftLayer and a small pile of others.

It's free and includes benchmarking and pricing data into it's recommendations.

http://www.oncompare.com/categories/cloud-computing/decision...

How does Rackspace compare with say Joyent? I'm cross shopping for some more CPU. I've been a very long EC2 user/developer.

I've used both and performance-wise my team didn't see any drastic differences. We switched to EC2 mostly because of RightScale (which now also runs over top of RackSpace). It's not cheap, but running on RightScale has made server maintenance and scaling a breeze for us. If this is a pain on your team, I recommend them.

Does anyone have experience with http://www.cloudsigma.com/? They seem like a nice alternative for european companies. It would be interesting to see a comparison with Amazon and Rackspace.

http://bit.ly/cWySC0

Miha A.

Please have a look at:

http://www.cloudsigma.com/en/our-cloud/how-we-compare (feature comparison)

https://cloudsleuth.net/web/guest/global-provider-view (select Europe for a comparison of our cloud performance against other providers)

I'm considering a server provider for a new startup I'm working on, but the recent EBS issues of Amazon have put me off it entirely. Would you suggest I reconsider? What are your experiences with AWS?

That is the exact reason why I am cross shopping as well.

You also have the option to use ephemeral storage if you prefer not to deal with EBS. All instances provide ephemeral storage with the exception of t1.micro.

If he's dealing with EBS he probably doesn't want to use ephemeral and the fact that not persistent.

This is the issue I am debating. Amazon offers a nice suite of services but the EBS issues are giving me reason to pause.

wait a minute, i've been using a fedora instance on rackspace with 256 MB memory, it's much less than $60 a month.

Just guessing here, but the reason Rackspace Cloud is probably more performant than EC2 is probably because EC2 is more utilized than Rackspace.

There was a comparison about a year ago that tested performance over a week span. Linode performed better than Rackspace Cloud and that fits my experiences. No Windows so not helpful to Dave but worth looking at.

http://journal.uggedal.com/vps-performance-comparison/

Isn't the word "cloud" is just an expensive way to sell shared hosting?

If you're on a shared host that provides root access, I suggest you find a new host.

no comments?

waaaait for it...

Reply on news.ycombinator.com