VM export for EC2 | Hacker News Reader

VM export for EC2

aws.typepad.com

90 pointsgtaylor27 comments

This is a useful tool, but spinning it as a victory against vendor lock-in is disingenuous. VM formats are well-documented and do not constitute a barrier to migration by any stretch of the imagination.

I would love to hear more about the real barrier to migration: the cost and latency hit of cross-cloud network communication. Who cares if you can move your app server to VMWare, if it needs low-latency access to 10TB of data sitting in amazon-us-east?

I helped to write virt-v2v[1], and one difficulty is changing the kernel config / Windows registry config / drivers / etc so the guest can be booted on another hypervisor (Amazon is Xen-based, VMware and KVM are completely different and need different drivers). You have to delve into the filesystem using libguestfs, and make some pretty complex changes.

As you say, the other difficulty is copying gigabytes of data across clouds. My colleague at Red Hat wrote snap[2] which can turn Linux guests into digests (basically a base distro name and list of packages and list of config changes), so this can be reduced massively. We're currently extending this work to support Windows and more Linux guest types.

[1] http://libguestfs.org/virt-v2v

[2] http://mo.morsi.org/blog/node/347

> the cost and latency hit of cross-cloud network communication.

AWS Direct Connect might help with that. Anyone have any experience with it?

"AWS Direct Connect makes it easy to establish a dedicated network connection from your premise to AWS. Using AWS Direct Connect, you can establish private connectivity between AWS and your datacenter, office, or colocation environment, which in many cases can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections."

http://aws.amazon.com/directconnect/

Well, it does move the burden for maintaining compatibility from third-party import tools / glue code / adapters back to the original vendor, which I think is extremely sensible.

Let's hope they support Linux exports soon then. This solves another problem as well, which is easily moving images around availability zones.

Imports, too - they launched an import tool about a year and a half ago, but it too only supports Windows.

We plan to support additional operating systems in the future, along with more virtualization formats.

moving images around availability zones

Presumably you mean regions here? Within any one region, you can use an AMI in any of the availability zones.

Whoops, yes of course. That's what I meant. Moving images between regions is currently a pain.

Agreed. Every time I publish a new FreeBSD AMI I spin up 7 instances and run a bunch of "dd | ssh | dd" to send the bits out to the right places.

At this point I have the entire process scripted, and it's still painful.

I was excited until I found out that its currently only Windows is supported :(

I'm a little unclear on the specifics of actually running a foreign VM on EC2 and vide versa. Don't AMIs usually contain special provisions for running on Amazon's infrastructure?

This is awesome - what would be the remaining use case/advantage for using an AMI when you can now snapshot to more widely supported formats and import them?

Edit: Oh, this is windows only, significantly less exciting. I wonder why linux isn't supported?

I wonder why linux isn't supported?

Linux runs on EC2 via Xen/PV, which is a kernel-hypervisor interface not provided by other (non-Xen) virtualization systems. Exporting linux AMIs would require reaching inside your filesystem and modifying your kernel, which is far outside of the scope of anything Amazon would want to do (both in terms of "really complicated" and "liable to accidentally break things"). The same applies to importing images, of course.

[I don't work for Amazon and I don't speak for Amazon. The above is speculation based on information published by Amazon about EC2.]

Kernel isn't a big deal, Colin. An entire filesystem should boot the same when exported. If you rsync / elsewhere, twiddle a few knobs like network config then reboot, you should be able to move an Amazon image anywhere. I have successfully done this between disparate virtualization providers, and Xen to dedicated.

Amazon is unique in that they use PV-Grub everywhere to boot a customer filesystem kernel (newer behavior), whereas on other virtualization providers like Linode, the kernel is selected at domain creation time by the hypervisor, and loaded from the hypervisor's filesystem (not the customer). You can use PV-Grub but it's opt-in, not opt-out.

I have successfully done this between disparate virtualization providers, and Xen to dedicated.

I'm betting you were running Xen/HVM then. (Most people do, these days.)

Well, yeah, but it's the filesystem, not the virtualization tech. You could rsync a dedicated to a dedicated and it'd be fine.

It's not really complicated. If you have, say, a RHEL 6.2 image booting on EC2 you can take the same root filesystem and boot it with a kernel from any other RHEL 6.2 installation with some editing of the grub configuration.

But you're right that Amazon isn't really interested in doing that work for you. Linux people who really need this are no doubt already doing it.

Sure, but if you're running a custom kernel you'd need to run a custom non-PV kernel. And there's no remotely sane way for Amazon to figure out what customizations people have made to their kernels.

How about making it easier to get the sources of the packages (for Linux) so that things are easy to test locally?

Last time I checked (admittedly a little while ago), one had to do API calls through some convoluted way.

I was tempted to script it.

I'd like to see the dependence on PV-Grub lifted.

Maybe Amzn is just not too skilled working with various bootloaders?

What's wrong with PV-grub?

Nothing, if you like GRUB. What if I don't want to use GRUB and I just want to use the OS's native bootloader?

Then don't use paravirtualized Xen. Other boot loaders don't understand Xen/PV.

Does "other boot loaders" include NetBSD 5's bootloader?

I believe so, yes. It certainly includes FreeBSD's bootloader.

Reply on news.ycombinator.com