Quick Redis Benchmark: AWS vs. Azure | Hacker News Reader

Quick Redis Benchmark: AWS vs. Azure

zapier.com

Having used their Linux offerings in 2012 - the only thing that I can say is whilst Azure does provide good performance for the price. You lose out on stability, sanity and on choice.

I know the cloud is supposed to be a place where the servers are volatile ... BUT ... random reboots with non-writeable disks when they come back as well as network connectivity problems between servers are not something that I've experienced at Rackspace or AWS.

The support at Azure is pretty laughable ("what's a public key?" from one of their so-called TCP/IP experts) if you manage to get a phone call with the folks over at Redmond. Otherwise all you have is forum support where you are basically treated like an idiot and blamed for their mistakes unless you can prove otherwise.

The Windows offering is a little better, more stable for sure - but, then you run into crazy firewall issues if you are using something like AzureRunMe to do the work and set-up bespoke web app servers.

For what i spent the best part of a month failing at getting set up at Azure, took me a few days to set up at Rackspace, with configuration management and service monitoring.

I wont be rushing back to use Azure again any time soon and be warned that this benchmark is only a small glimpse at what is otherwise a mighty shit sandwich.

To be fair, Azure IaaS was in preview in 2012. It didn't become generally available until April of this year. http://blogs.msdn.com/b/windowsazure/archive/2013/04/16/the-...

totally aware of that, that doesn't discount from what was in my opinion truly terrible customer service and knowledge of their own products by the engineers. it left me with a really bad taste in my mouth.

I can't comment on linux but I've had no issues with Azure using windows, in fact I would say its fantastic. I been have using cloud services web/worker roles, messaging, blob and SQL azure too... I have been also integrating cloud TFS publishing, and automated deployments. Its been a dream so far. I am looking at the elastic scalability block also for scaling out and in, it all looks great.

I mean transitioning from staging to production is just a virtual IP swap, sweet...

So if you get your setup right and go with the Microsoft cloud setup I think its smooth sailing.

Check out http://metricshub.com (acquired by Microsoft) that provides a great dashboard of billing usage, but also lets you auto-scale resources, all for free.

This looks very good, will definitely try it out..

No offense here, but how can you judge on Azure utilizing a project, that is not even an officially supported Microsoft product? There are other ways to deploy JBoss or Ruby on Rails on Azure. Give Google a try :) I am using Azure for over a year now, without running into serious issues that could not be solved with Stack Overflow or similar sources. Never had to use the official support. Azure performance is great and solution development is backed by a great set of tools. That's my personal experience.

Another recent benchmark:

Cloud Server Performance: A Comparative Analysis of 5 Large Cloud IaaS Providers

http://www.cloudspectator.com/cloud-server-performance-a-com...

Interesting that Windows Azure comes up at the top not only purely in performance but also in the price performance ratio as well.

Indeed, they're matching AWS' prices for IaaS, at least on pay-as-you-go. For reserved instances there seems to be a difference.

Color me impressed, 3x throughput for the price(60,000vs20,000) for azure might be enough for me to create an account. Can anyone comment on what accounts for the differential? It doesn't seem like the hardware would do it,but I could be wrong.

EC2's virtualization is pretty terrible for Redis http://redis.io/topics/latency

The newer m3 instance types are better, but for the most part Redis is scarcely usable on EC2.

I wouldn't go as far as calling it scarcely usable, but EC2 does add some overhead to running redis. I've been using it on EC2 for over 2 years (some of the metrics in the doc you linked are on my machines BTW :) ), and you just have to find workarounds for persistence because of the high fork times - it adds a bit of complexity, but a dedicated slave for saving is not that big a deal, and can reside on the same instance as the master).

We've recently moved to m3 instances and the problem has completely gone away, hopefully Amazon will migrate all instances to use HVM.

Try linode, numbers are going to surprise you.

The AWS medium tested is a single core. The Azure is dual core. Context switching could account for the difference in preformance.

Just because they are both labeled medium doesn't mean they're comparable. The Azure is also about %50 more the AWS last time I looked..

Edit: I was wrong about the pricing. They are similarly priced.

They both cost $0.12/hour unreserved, so it's a fair comparison if you are thinking performance per dollar.

http://www.windowsazure.com/en-us/pricing/details/virtual-ma...

Azure is listed at $0.18/hour.

Edit: My mistake. Windows is $0.18/hour, Linux is $0.12.

Edit2: Now that's interesting, Microsoft charges more for a Windows VM.

It's an honest approach: Windows VMs include a license.

You'd think that Microsoft wouldn't have to license their own software.

It would probably be seen as anti-competitive if they licensed between business units for free.

Performance/dollar:

Azure wins - 3.1:1 on demand

Azure wins - 3:1 on 1 year reserved (author is incorrect here - does not account for initial reserve payment)

Less noisy neighbors? I am not sure. Seems like a good deal though.

> The AWS medium tested is a single core. The Azure is dual core. Context switching could account for the difference in preformance.

Redis is mostly single-threaded.

http://redis.io/topics/latency

the benchmark still has to run somewhere :) and it takes up CPU as well.

running both the server and benchmark on one CPU reduces performance dramatically. On a 2 core EC2 machine the results are much closer.

However if you ran the benchmark from a remote node, you can really compare apples to apples.

But given that they both cost more or less the same, having one CPU still sucks just the same.

http://redis.io/topics/latency : Redis is mostly single threaded since actually from Redis 2.4 we use threads in Redis in order to perform some slow I/O operations in the background, mainly related to disk I/O

Could this affect the benchmarks then? I don't know if it does, but if the benchmark is also writing to disk, wouldn't a dual core machine perform better?

Reply on news.ycombinator.com