Announcing VM Import for Amazon EC2 | Hacker News Reader

Announcing VM Import for Amazon EC2

aws.amazon.com

105 pointsjonmc1235 comments

Amazon continues to amaze me with what they are setting up.

I've had my problems with some of their policies, but overall they are really doing a bang-up job. They are doing what any good business does: they are making it really hard to pick any other alternative.

I really like the idea of moving VMs in and out of the cloud. Can't emphasize enough how neat that is. If other vendors start implementing this, we could have true infrastructure portability. Very cool.

One problem I've had at every place employment I've worked at in the last few years is getting a VM which matches production to use as a sandbox. With this kind of thing it would make that a snap. "What's the difference between this and production?" "Nothing. It's the same image."

Yep

Not only that, but many enterprise systems are not just a server, or even a couple of servers, but a cluster of servers of various configurations.

With good bandwidth, you could pull the entire bunch down locally to use in dev and testing. That's unheard of as far as I know (although it has been the holy grail for some time) Businesses that start and grow in an environment like this are going to have significant advantages over older businesses.

VMWare offers a tool that solves this problem -- Lab Manager (http://www.vmware.com/products/labmanager/).

It even supports setting up environments that contain isolated clones of multiple servers (see other child comment).

From what I understand it's going to be replaced with a new product fairly soon, but I've heard great things about Lab Manger.

Check out DevStructure blueprints (http://devstructure.com).

> I really like the idea of moving VMs in and out of the cloud.

Out of? That's funny, I don't see a "VM Export" feature. Surely you don't mean booting up a debian rescue image and DD'ing your disk over SSH somewhere else?

It would be great to one day be able to take a live VM, package it up and transfer it to your laptop for testing, then you can patch things up and send it back to EC2, but that day is not today.

I'm sick of having these cloud services have such blatant disregard for switching to a different provider. It gives a new, colder meaning to your statement: "They are making it really hard to pick any other alternative."

Uhh, what about ec2-download-bundle? Last I checked you could use that to download your image files.

http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineR...

I am not so sure the exported bundle can be run off AWS, are you sure about that?

On a side note, doesn't the download bundle task require you to stop the instance? That could be a problem since the task can take quite some time to complete during which time your server will be unavailable.

Yeah. It's just a Xen VM. I do this every month or so. You just have to make sure that you have a kernel and modules you can boot the image with on your local Xen host. This is pretty easy, you can do it with whatever package manager your distro comes with, or if you created the image on a local Xen machine, you probably already have this. (yum install kernel-xen works on rhel)

It's true that you can't really do this with a live VM, since the AMI on EC2 will be whatever you had on the image before you last booted it. But you can do a ec2-bundle-vol on your live image to create a new AMI, then do your ec2-download-bundle on your new AMI.

You're really not as "locked in" to EC2 as some people think. You can easily pull your images off there, and then install Nimbus or OpenStack or something and run a local cloud. That's what we do at my job.

Hopefully VirtualBox support will be added at some point. It's amazing what Amazon is doing with AWS.

They announce it when only support Windows Server 2008 SP2? Seems rather underwhelming when that is all you can import.

Building an AMI or cooking with chef/knife is easy enough, but I know a lot of people who'd like that kind of simplicity. Just too bad it is only one OS.

It's just a starting point.

And I bet they focused on Windows because there aren't as many great provisioning tools as there are for Linux.

Out of curiosity, how do people currently build a custom new EC2 image? Spin up a base instance and then run a script?

If you're using Ubuntu, they have a tool for creating new EC2 images called python-vm-builder-ec2.

There's in fact a whole command line suite, which you can script anything through.

We start with a base image and then use puppet to manage anything that needs to be installed/setup.

I don't understand why more people don't use bcfg2. Its awesome for this purpose (and I think better than puppet or chef.)

Then educate people! Don't just say it's better, say why!

I did a midrange deployment with bcfg2 (~250 machines), and what I found is that it was slow to use.

Firing off a bcfg2 push would take almost 20 minutes, which was pretty unusable; The big problem was that it didn't handle large numbers of connections simultaneously well, so I ended up having to stagger them.

CFengine was able to do the same push in < 30 seconds. Win.

I'll be looking at Chef again in the near future. It's got a LOT of nice features; Hopefully we can make it scale out quickly.

yes, that is what i've done, or rather just log in, set things up, and then save a snapshot, repeat as needed. not sure this will speed things up in general as far as iterations, but may save the $0.10 you may spend setting up a workable vm.

I use a custom AMI. I started with a base image a long time ago. Since then I have been upgrading and tweaking. There are several tutorials on creating the image. API tools are available for the whole process if you are on an instance-store image. It creates an image of the instance filesystem in /mnt and then breaks it into chunks to upload to s3 with a manifest and registers the manifest as a new private ami.

Knife and Chef: http://www.opscode.com/chef

I'd like to see export.

Interesting that they support importing VMware images - I was under the impression that EC2 used Xen.

they do. the import just changes the virtual disk format, the same way vmware converter can import xen images, MS hyper-V images, MS virtual desktop images, etc.

Converting between image types isn't too big of a deal; qemu-img can convert between most of your common on-disk formats (including VMDK), and wouldn't be hard to extend to whatever they're using behind the scenes.

I wonder if I can convert my old dusty Win XP Parallels image and VM-import it into EC2... That would be pretty cool!

We support specific versions of Windows 2008 SP2 at present, with plans to add other versions and other OS's over time, with a focus on server OSs (as opposed to desktop).

This is an amazing change that hugely reduces the cost of transitioning to AWS. We're deploying VMs on Xen at our university and have talked about AWS but there's a high 'coefficient of static friction'. This could change that whole discussion. Well played.

Does that answer my question? IIRC Windows Server 2003 was quite similar in HW reqs to XP, so should 2008 support it?

VMDK is an open (GPL compatible) specification used by a number of different VMMs.

Huh? What does it mean when a specification is GPL-Compatible? I was under the impression that an implementation of a specification was GPL-compatible or not.

Good question. First, a specification is technically under copyright and GPL doesn't just apply to software so the GPL should probably be able to be applied to non-software, but the following wording from their press release makes me think that it also covers implementations of the spec:

"VMware, the global leader in virtual infrastructure software for industry-standard systems, today announced that its virtual machine disk format specification for defining and formatting virtual machine environments is now openly available, downloadable and free of charge. This will enable use by all developers, software vendors and projects and includes open licensing compatible with those operating under open source licenses such as the GPL."

Though it's less than widescale support: The VM Import process currently supports VMware VMDK images for Windows Server 2008 SP2.

Beautiful

Reply on news.ycombinator.com