Performance comparison between EC2, Slicehost, Linode, Rackspace Cloud, Prgmr
journal.uggedal.com
journal.uggedal.com
If the latter, a zero origin is fairer. While the day-to-day differences are of some interest here, the main interest is provider vs. provider, so the axes are not ideal. They make the provider-to-provider differences seem larger.
If you're running a web app, my bet is you'll get a much better ROI on time if you pick an expensive DB query you're running and try to analyze/optimize it.
All that said, if you haven't yet picked a host, this does offer some interesting insights. Still, I would rank APIs, GUIs, management panels, and helpful support as way, way more important than raw performance.
From what I know linode is also cheapest for bandwidth overage. Far far cheaper than slicehost.
I have had no issues at all. Highly satisfied customer here.
(the vast majority of or problems are provisioning-related, as well... we're working on that. But still. If you didn't get the refund email me and I'll see to it.)
Very minor issue when I first signed up, but I hadn't even logged in to the box yet. I emailed Luke and in about an hour I got a response back saying it was fixed.
I'm not using it for anything critical, so if something goes down or seems not correct, I might email and ask what's up, but usually I probably won't even notice or care enough.
I also follow his twitter, which he seems to keep updated if something is not functioning as it should.
Edit:
I've been using it for a few months and there was only been one instance that I noticed where things were not working as they should.
I emailed and got a quick response back that he was working on resolving the issue.
So, it's a little unfair to do a benchmark when 95% of the system's resources are taken up by an unknown factor.
(Note: I don't host with any of these guys, so I'm not defending Slicehost or anyone else. In fact, I run my own hosting company and I'm working on our own cloud offerings right now.)
http://journal.dedasys.com/2008/11/24/slicehost-vs-linode
Looking more at memory and price, and Linode is a winner there, too.
It's a pity that Slicehost doesn't offer a 32bit system, as that would negate most of the difference, and they are good people with a good product. But I don't want to pay extra money for fat pointers.
I'd speculate amazon has many different generations of hardware deployed and the EC2 Compute Units do not really capture the differences between these, particularly when it comes to memory bandwidth.
Do they have good DNS tools like SH?
EDIT: Ok, I just found some availability information in their FAQ:
"What we can boast about is our commitment to resolving these issues in the quickest fashion possible. Most customers will tell you the last time they rebooted was to take advantage of a plan upgrade. 99.9% uptime, or your lost time is refunded back to your account. "
This essentially means it's worthless: they're only backing it up with paying you back the time your server was down (they'd better do that!). Looks like they don't have much confidence in themselves in that area.
Here’s our SLA: we’ll do our best to keep your machines running smoothly for as long as possible and get them up ASAP should something go wrong.
If (say) the NYSE was negotiating for a host, they would make sure they had an SLA that gave massive damages if the host failed.
But for commodity hosting, the provider writes the contract. There is no way they are going to expose themselves to your losses other than a fixed refund for the time you lost. Not just because of the costs, but because the costs would be a PITA to calculate. If Twitter goes down and alienates its customers, how would you calculate the damage?
You don't ensure uptime with SLAs, you pick good providers and go for redundancy.
Well, my point is that this money back is basically worthless: say, at some point, your server was down for 4 hours, and you pay $100 / month. This means you would get $0.55 refund for 4 hours of downtime. I'm sorry, but to me that sounds like "we guarantee this amount of uptime, and we back it up by absolutely nothing!".
Not that I expect my damages to be repaid, not at all, but a webhost's availability guarantees are only as good as the money they put in to back it up. If they would, for example, promise to pay back a month of service fees, it would not recover my damages, but at least shows they're a lot more serious about the guarantees they're giving.
Anyway, I do not have anything against Linode, not at all: it's just that I'm wary whenever a webhost is this unclear about availability guarantees. I probably had a few too many bad experiences in the past.
edit: Sorry, someone below has said similar thing.. should of read the entire thread first!
And you know what's great about them, they have been quietly supporting the Rails Rumble community and it's time we show them our support for being best at what they do!
Look at the figures. The performance of Slicehost follows a sawtooth like pattern. The quantity standard deviation is useful because it quantifies what to expect. Plus or minus one standard deviation means that ~ 2/3 of the time you will fall in that range.
If you think about the problem a little bit, you might be more worried about the standard deviation of the standard deviation. This, in fact, would be a useful quantity, but hard to measure.
EDIT below this line ------- Several comments below have commented that SD is somehow less useful if it's "large" (or large relative to the mean, or whatever). The reason people think large SDs are indicative of a poor experiment is that in school lab classes one calculates the SD and call it the "error".
The standard deviation is a measure of spread, if it's large then the spread is large. Knowing the spread has value. In this case, under the parent's experimental conditions EC2's performance is more constant than that of slicehost's.
A fair critique of the blog posting is that the error on the standard deviation may be large, depending on the experimental conditions. It is _not_ a fair critique to say that the SD is too high to make a prediction, you just have larger performance spread. Note that the performance spread described is not necessarily "error". The spread is inherit to either the server (as implied by the article) or the method (in which case it is an error).
As a beginner, Slicehost was really great. The control panel is really straight forward and easy to use, their support is quick and helpful, and their how-to articles are really great for beginners.
To quickly compare the how-to's in a similar category: http://articles.slicehost.com/ubuntu-hardy vs http://library.linode.com/lamp-guides/
So they both have articles for the absolute beginners, but it still seems like Linode slightly favors the more intermediate to advanced users in their how-to's. Purely subjective, but Slicehost's seem a bit easier to read / follow.
That being said, this article was incredibly interesting. Now that I have a bit more experience with VPS's and Unix, I am very tempted to head over to Linode.
I am still developing my web app on Slicehost but I have not seen any performance problems since I have very little traffic. I am now considering setting up the production server over at Linode when I'm ready to go live.
Thanks for the great article.
Provide network latency/tput data also, please. :-)
Checkout http://www.hetzner.de/en/hosting/produktmatrix/rootserver-pr...
I'm a happy customer
Considering how little I paid for my 512M prgmr VPS, I'm very pleased to see these results!
http://book.xen.prgmr.com/mediawiki/index.php/SLA
it seems like Prmgr is ran by a single person, with a 99.5% uptime guarantee. I bet this reduces costs quite a little.
Also, I build my own servers, which saves me some bucks, but it's not orders of magnitude.
uh, also, compare my SLAs to the other providers before you start saying it's weak. (I mean, my sla is weak, I agree, but so is everyone else's.)
I was once contacted about an old ticket and asked by Will if it was solved, since it was solved on their IRC channel (and was not marked as solved maybe). It shows that they make sure they respond to all the tickets. Really sweet of them to do it.
I agree with mahmud. Same here friend, I've been that "walking PR agent" for prgmr ever since I started out with them.
Besides Prgmr is for those who like the real raw stuff - bare bones VPS with shell access.
The SLA is always something you don't want to read with any hosts :P Coz it includes all the bells and whistles they say they would provide you if they go down,and that would sound like they would go down :)
There were 3 reasons why I use Prgmr: 1.) They support Paypal (yes, I cannot use a credit card since I'm not yet eligible to have one in my country) 2.) The pricing is great 3.) They have this small 128mb pkg on which I can develop and test my app. So when launching I can upgrade my plan. Thus saving me a few dollars which might sum up to one more month's payment when I launch :)
as far as I can tell, most of the competition has not been lowering their prices by much over the last few years. (linode has some.)
Also, to be fair, if you pre-pay for a year, ec2 gives me a run for the money; I only have a 20% discount if you pre-pay for a year.
I'm not switching because they meet my needs, my customers are happy, and switching would take me lots of time that I can more profitably use promoting my software. I don't get paid for doing work, I get paid for solving problems. Slicehost is not a problem for me.
Your mileage may vary.
It's not about 'being a problem', it's about optimizing when there's something better/cheaper available.
I do, however, also have a Linode VPS and will probably go with additional Linode VPS' in the future if the service continues to be stable.
In my humble opinion, Slicehost offered a great deal two years ago but they should adjust their plans in the future or customers will move to competitors.
My understanding is that Amazon is competing on developer and sysadmin productivity with the depth of their API and focus on manageability-- not on performance. Is that correct?
JSON REST goodness.
At one point I had the Slicehost crew set my account to provision slices in their newest datacenter in Texas. Things seemed snappier after that, though there could be several reasons why and it's hard to know if it was just a matter of newer hardware running the Xen dom0.
----
processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : Quad-Core AMD Opteron(tm) Processor 2350 HE stepping : 3 cpu MHz : 1994.999 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch bogomips : 3993.30 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate
processor : 1 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : Quad-Core AMD Opteron(tm) Processor 2350 HE stepping : 3 cpu MHz : 1994.999 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch bogomips : 3993.30 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate
processor : 2 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : Quad-Core AMD Opteron(tm) Processor 2350 HE stepping : 3 cpu MHz : 1994.999 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch bogomips : 3993.30 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate
processor : 3 vendor_id : AuthenticAMD cpu family : 16 model : 2 model name : Quad-Core AMD Opteron(tm) Processor 2350 HE stepping : 3 cpu MHz : 1994.999 cache size : 512 KB fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu de tsc msr pae cx8 apic cmov pat clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch bogomips : 3993.30 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate
In my humble opinion, at least some of these benchmarks (the CPU ones) do not represent anything, as they depend on the load of neighbor instances on the same physical host. Once you see that, how can you possibly make business decisions based on these benchmarks?
Color me surprised.
I'd like to see a more complete comparison. I personally use Rimuhosting and Rackspace, I'm happy with both.
If so, I can see myself getting Linode for my next project. Heard a lot of good things about them in the past.
In defense of slicehost, they have a no-bs interface with good reading resources to setup machines and very fast and helpful customer service. I was once more than 10 days late in payment because I had problem transferring my money to the account I exclusively use only for online payment and they were very reasonable with me.
People are already starting to do the cloud->user (http://www.apparentnetworks.com/CPC/scorecard.aspx) but I think the other stuff is equally important/interesting.
[edit] lol. forever must = 3 minutes in my world... loads afterwards.
I think too often we lay blame on websites when usually it's a user error.
If he put an Amazon medium or large instance up against those others, it would have fared much, much better.
For example the EC2 instance storage is known to be dog slow. I assume that's the main reason for it looking so bad in your graphs - the picture might change with EBS.
As others have pointed out, synthetic benchmarks are a tricky beast generally. For fairness you should have optimized each host to their max potential - because that's what a regular user would do. But still, the variance in cloud-hosting makes it difficult to obtain representative figures. Slicehost might just have looked awesome in a different week...
Edit: I think you should point out, when you edit your original post, where you only mentioned Linode not slicehost. Thanks.
In the same way I never batted an eyelid with David Welton's Slicehost vs. Linode[1] article. It was informativve and unbiased and I was happy to provide some incentive back for that.
[1: http://journal.dedasys.com/2008/11/24/slicehost-vs-linode ]
I (and most others hopefully) already knew about prgmr and EC2 has horrible performance, even before I saw this benchmark, the only one new to me is Rackspace performance numbers (BTW Rackspace owns slicehost).
I don't see any logic behind your objection.
Even with no conscious/devious intent, this sort of interest can bias the results. For example, what if the relative results could vary arbitrarily from week to week (especially possible with the neighboring load on VPS nodes)?
If you get a plausible result that shows the companies pay commissions do best, you are incented to publish right away. If you get a result that shows the non-commission-paying companies on top, you will dig deeper: maybe there are ways the commission-payers are better, in a longer analysis.
(It's similar to the publication bias in other research, which also requires no scheming intent, just natural behavior in response to incentives which are each alone reasonable.)
However, in this case, the author is not sponsored by or works for any VPS service. He provided affiliated links at the end of the article, where one of the linked service did not prove to be superior than the other according to his analysis. If he sneaked in an affiliated link within the article, the objection and skepticism could be somewhat justified (still I think its somewhat fair game).