Hacker News Reader

Comment by kartikkumar | Hacker News Reader

I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the wiser? Am I expecting too much?

View on HN

EC2 instances don't come with vmotion. It's up to the customer to detect a failed/retired node and restart on another EC2 instance.

The first thing you discover when reading through the various options is that you need to treat ALL local storage like /tmp, subject to deletion at will. Keep your persistent storage on EBS/S3.

And even if you do keep your important stuff on EBS, make sure you take snapshots on a frequent basis. We have received this email a couple of times:

    Your volume experienced a failure due to multiple failures of the 
    underlying hardware components and we were unable to recover it.

    Although EBS volumes are designed for reliability, backed by multiple 
    physical drives, we are still exposed to durability risks caused by 
    concurrent hardware failures of multiple components, before our systems 
    are able to restore the redundancy. We publish our durability expectations 
    on the EBS detail page here (http://aws.amazon.com/ebs).


    Sincerely,
    EBS Support

Fortunately, we had recent snapshots and it was a matter of (manually) spinning up a new instance from those.

Edit: proper quotation

Windows Azure actually does this. If the host your virtual machine is on for some reason fails or needs to be replaced your entire VM is migrated to another host. The migration process can take a few minutes but all your data is safe.

My point being. On this topic AWS could learn from Microsoft on how to do cloud.

The one problem they have is that the majority of their instances include local storage, which would make migration impossible. So the best they can offer is a reboot so the server ends up on another host.

They could potentially do this on their second generation (M3) instances, as well as micro instances if they wanted to. However I'd guess that these instances are just a small percentage of the overall servers used.

> The one problem they have is that the majority of their instances include local storage, which would make migration impossible

True for AWS. Using VMware vSphere then this could be done with a shared nothing migration which moves compute and storage (vMotion + storage vMotion combined).

Xen also does this, but AWS may still be using a very old fork of Xen 2 or 3.

It's unfortunate that there doesn't seem to be a large-scale consumer for the XenServer platform now that Citrix open-sourced the entire thing.

While Xen should make live migrations technically possible, it would probably reduce EC2's provisioning flexibility and introduce undesirable complexity.

Migrations would be restricted to hosts running specific releases of the hypervisor [1], and AWS's SDN systems would need to handle these changes in very-near-realtime.

[1] wiki.xen.org/wiki/Xen_Version_Compatibility

Reply on news.ycombinator.com