Hyper – Hypervisor-agnostic Docker engine | Hacker News Reader

Hyper – Hypervisor-agnostic Docker engine

hyper.sh

125 pointstbronchain78 comments

I like this idea. It's a stripped down VM capable of only running containers. There are a couple of downsides, compared to straight docker though - you have to go through the VM layer for disk io. So, if you have a shared volume, there will be a performance hit.

However, you get a big boost for security.

This type of tech could make it possible to use (Docker) containers in a true multi-tenant system.

One question though - is it possible to specify the vmem and vcpus for the qemu VM? (Either from the command line or pod file?)

Hi, Thanks for your interest, and congratulations, you found the original motivations behind Hyper :)

To anwer your question, yes, you can specify them. Check out the podfile documentation to know how https://docs.hyper.sh/reference/podfile.html

Yep - there it is... I missed that when I first looked.

Thanks!

As an aside - do you have a feeling for how much processing CPU power or io speed is lost when running in the hyper VM?

here you go: https://hyper.sh/faq.html (performances section)

Q: What are the performances of Hyper?

Boot: Start a hyper Pod only takes less than 350 millisecond. CPU: CPU Performance overhead is about 0.4 percent. Mem: Memory Performance overhead is about 0.64 percent.

Containers have been used in multi-tenant systems for years in both Solaris and FreeBSD via zones and jails

Not sure how secure they are. But sounds a bit scary to run my app in a shared-kernel environment.

Docker also uses a shared kernel...

edit: and why is it different than a shared hypervisor? Hypervisors have exploits, too...

Yes, but the nature of lxc makes it much easier to exploits.

And on every Docker thread someone says this. I know I've said it myself.

However, Jails and Zones aren't a security panacea either. Especially if the untrusted user has access to both the Jail and the host system.

The use case I'm thinking about is on a multi-tenant HPC cluster. There are very few non-Linux clusters like this and all of them require strict security. I'd love to deploy docker containers for data processing, but so far it's not been possible due to security concerns (unless running through a VM first).

However, I went ahead and made it explicit that I was referring to Docker containers.

Why would you give a user access to the host environment AND the jail?

There are more use-cases than just cloud web apps.

The case I mentioned was an HPC cluster. There is a ton of interest in using containers more in HPC, just for the flexibility in administration and versioning aspects. But there is a catch - you need to ensure that each user is completely isolated from the others, users can't have root access, and they can't escape from their containers. These users need to have access to both the host and the jail for practical reasons (mainly due to the way the common batch schedulers end up working and data/storage access).

Thus far, those restrictions have made it difficult to deploy Docker (or other containers) on clusters. It is possible to deploy VMs with some schedulers, but it largely hasn't caught on in many places.

Agree, and I'm a huge fan and user of both.

But separation enforced at the hardware level is a different, unambiguously better, thing.

I am a non-fan of Docker, but this elevates it to a level of strong contention. And brings Linux along for the ride.

Would be nice to have this on Macs too no more crappy VirtualBox ;-)

You can do Docker with VMWare using Docker Machine:

docker-machine create -d vmwarefusion --vmwarefusion-boot2docker-url https://github.com/cloudnativeapps/boot2docker/releases/down... my-docker-machine --vmwarefusion-memory 4096 --vmwarefusion-disk-size 30000

it seems very similar to clearcontainer that was announced a few days ago, i.e. interesting since both are much nicer than "traditional container" solutions.

Now for the interesting stuff.. init process (C): https://github.com/hyperhq/hyperstart

Daemon (Go): https://github.com/hyperhq/hyper/tree/master/hyperdaemon

Stuff to replace with overlayfs ;) https://github.com/hyperhq/hyper/tree/master/storage/aufs

yes, we do not support overlayfs yet, but will. overlayfs is the future of unionfs, and aufs is the current. :)

Looks interesting. How is this different from Clear Container several days ago?

Hi there,

Thanks for your interest in Hyper!

We also noticed the release announcement of Clear Containers a few days ago. There are 2 main differences, the first being on the technology used, CC using RKT, while Hyper is based on Docker images. There is also a difference in term of philosophy, where Hyper aims to be a technology-neutral open source solution.

Hyper also have more features, such as running a Pod rather than one image on a hypervisor as a schedule unit.

For more details, please check our FAQ (https://hyper.sh/faq.html)

I don't feel this is correct. rkt can run docker images[1] and once the clear containers patches[2] land you will be able to run a docker image or an appc image inside of a VM under rkt.

[1] https://github.com/coreos/rkt/blob/master/Documentation/comm...

[2] https://github.com/coreos/rkt/pull/946

I think the question was specifically in the context of CC.

Have we reached Peak Container yet?

Hi there, I am not sure what do you mean by "Peak Container"?

Sorry, it's a pun on "Peak Oil" (the idea that at some point we're going to pump and all the oil in the earth will be gone).

Not a dig at Hyper in particular (which looks cool), just that the "containers as service configuration management" market is extremely saturated right now.

The idea behind hyper is to propose something slightly different, answering the security issues you can have with containers.

We need containers within which we can run different hypervisors.

Its turtles all the way down....

Oh it may peak as a buzz word, but I think we'll be using them for a long time.

Are there good management and orchestration tools for Xen (or other hypervisor)? One of the things about Docker is that I'm having trouble finding good tools for managing an infrastructure of Docker containers. I know this is being worked on, yes, but it's early days and I need something that is mature.

If anyone can suggest Xen admin stuff, I'd really appreciate it.

There's xenserver (tools) which was opensourced a little while ago.

Most configuration management systems also have modules for managing xen. At the moment I'm writing some CFM code using saltstack with the xapi and virt for my own stuff, as I'm not a huge fan of the xenserver and xen cloud platform abstractions.

libvirt?

Is this just a cross platform boot2docker?

Hi,

Hyper and Boot2docker are two really distinct products.

The difference is, one (boot2docker) let you run containers on not compatible OSes, while the other one (Hyper) let you run each container, distinctly, in a dedicated VM.

Hope that helps :)

Sorry, but I don't quite get it. What's the advantage over having a dedicated VM per container?

Speed and resources https://hyper.sh/why-hyper.html

Isolation, and security. As @mbreese perfectly said, "This type of tech could make it possible to use (Docker) containers in a true multi-tenant system." With Hyper, all your containers will run their own kernel, instead of sharing the host's one.

It sounds like the shit Microsoft was gibbering about a little while ago, about how they would integrate their HyperV technology into their docker runtime

Hmm, I happen to know some details of that. It is actually a quite powerful technology with some out-of-box ideas.

Hmm, something seems strangely familiar about this code.

https://github.com/hyperhq/hyper/blob/master/storage/aufs/au... https://github.com/docker/docker/blob/master/daemon/graphdri...

Luckily, the Apache Public License is compatible with itself.

Hi. Indeed, we are using part of Docker's code as a base, as we are working on the interoperability between both.

That looks very interesting, I'm wondering how "container management" will be handled and how scalable is it?

Hi there, Thanks for your interest :) Container management is handled the same way it is in Docker (Hyper directly uses the Docker hub, and you can directly `hyper pull` Docker images). About scalability, it is, again, very similar to Docker's scalability. Hyper isn't compatible with some Docker components such as Compose or Swarm yet, but it is on the roadmap. You can try Hyper following the installation procedure: https://docs.hyper.sh/get_started/install.html

Is this just a stripped down type-2 hypervisor guest kernel, or is there something else going on?

Hi,

It does not run a full guest OS on the hypervisor, instead, initrd itself launches a group of docker images, aka Pod on it.

I like what you've done with Pod. Remember rkt has a similar concept, not sure why Docker doesn't have it.

So you're running all the containers on the initrd kernel?

I had the same question, and had to dig a little into the Google Pod documentation for this handy explanation: "In terms of Docker constructs, a pod consists of a colocated group of Docker containers with shared volumes."[1]

Based on the above tidbit, each Pod contains a few containers, thus a Pod (and hence all of the containers in the Pod) will indeed be running on the initrd kernel from the HyperVM instance.

The key point is that this applies for each HyperVM (i.e. one HyperVM launched per Pod). It seems the idea is that wherever you'd normally launch a container, you can now instead launch a lighweight VM that behaves like a container (i.e. fast to launch & small footprint) with default virtualization benefits.

Basically, if you replace all instances of the term "HyperVM" with "ContainerVM", it may help to parse their explanations more easily.

[1] https://github.com/GoogleCloudPlatform/kubernetes/blob/maste...

A pod on a kernel/initrd, contains one or more docker images in different mount namespaces. The "containers" in pod share network and ipc namespace each other.

The first you see when you load that page is a website telling you to curl to bash.

The next thing you see is a gif that didn't seem to tell me what it actually is.

I notice that it uses Kernel 4.0.1 - which is great that they're using something modern, however 4.0.1 has a critical EXT4 bug that causes major data corruption. This has been fixed in 4.1 RC5.

A lot of typos throughout the site too:

- "Subscribe to the lastest Hyper news." - " Pod is the first class in Hyper"

I won't even mention the use of comic sans on their diagrams...

> I won't even mention the use of comic sans on their diagrams...

Actually, you just did.

>curl -sSL https://hyper.sh/install | bash

Seriously? Developers need to stop sprouting this crap as an install method. Nobody in their sane mine should curl a script into bash to install a product.

That can be fixed in 30 seconds.

The product is impressive, and it would be respectful to communicate your valid message respectfully.

I think it would've been cooler if the url were https://install.hyper.sh

;)

you can try it now ;) cheers

There is no difference to running an executable. In fact, this is the BEST way to offer installation to users. He is literally showing you the source code so you can decide whether you actually want to run it or not. This stupid meme of not wanting to run scripts is just that: a meme.

How many production servers have you been responsible for in your lifetime?

"There is no difference to running an executable. "

... There are these following differences.

1. That url, assuming no malicious 3rd-party/nation-state is spoofing the response, could return any different version of the installer resource at any given time.

2. That url might not always be available, for any number of reasons, and how is someone who wants to "discover" this software when they are looking through their available package list?

3. Who knows what that url is "suppose to do" ... there is no signing process, peer review process, nothing, you get whatever the apache server on the other side of that HTTP request wants to give you, and your gonna send that right into your root shell...

4. Unlike a package, sitting in my personal safe, self host, audited, self-verified debian package repository mirror ... this URL might not work tomorrow, it might not work at 3:35am when my primary server took a shit and i need to rebuild the whole stack... who knows what this URL will do in between subsequent runs... it could return 2 different things when I am trying to build a cluster of this product.

0. Thousands. Tens of thousands, probably.

1. True of any download link as well.

2. See 1.

3. See 1, unspoken comparison to trusted package archives excepted.

4. Yes, getting your software into an official publishing channel is preferable, but not automatic, not immediate, and not without update latency.

I'm 110% with you on hating pipe to shell, however. Your arguments don't really address the issue.

And note also that you can just clone from github if you don't like piping to shell. And nothing prevents you from packaging it yourself in your own trusted repository. If you run serious infrastructure, you already do this.

Hi, you can view, download, and install from source code by checking out our github https://github.com/hyperhq/hyper

Even if you buy that encouraging users to uncritically pipe code to bash is a good idea - and I do not - this method of doing the curl/bash thing will very happily execute only a partial script if curl is interrupted in the middle.

So, no, it's very unambiguously not the "best" option, and maybe you should chill out a whole bunch, yeah?

While I'm not in the "'curl | sh' is evil" camp, it's certainly not as secure as installing a cryptographically-signed package (assuming you verify the signatures). HTTPS helps, but doesn't protect against a compromised server.

to be honest, without a one liner install method you don't reach people. most will blindly click this even thus its quite terrible.

certainly a better method is needed, but none of the alternatives provide "1 click install" like curl | bash, and as long as it's the case we'll see that..

After read through the install scripts, I admit I really like this method. Doesn't know why guys argues about that. If you need specific version or release, just checkout it out from github, after all, it's opensource man.

"Nobody in their sane mine should curl a script into bash to install a product"

This, so much this.

I was actually extremely excited over a similar product "flynn"... but they have also lost their mind when it comes to installation: https://flynn.io/docs/installation

> sudo bash < <(curl -fsSL https://dl.flynn.io/install-flynn)

Seriously?

How is that any easier than just providing a package for any given distro?

I mean, for fucks sake, just give me the URL to a tarball with a fucking Makefile in it. I can handle the rest, thank you very much.

The security concerns alone should force any sane system engineer to never pipe curl to sudo'ed bash process.

Would a zip with a Makefile do? Here: https://github.com/flynn/flynn/archive/master.zip

They have the source on Github, you're never forced to pipe curl to bash, it's just the default. Not a great default, admittedly, but hardly deserving all the hate.

How about contributing a PR with a script that produces OS packages, instead of complaining that the free cake doesn't come with a cherry on top?

I guess people are upset that this is the "officially sanctioned" way of installing the tool.

That's a serious bash script. That's the craziest bash script I've ever seen. It's like a full program. It is terrifying.

Just to counterbalance your fear: it's really nothing special or weird.

It looks pretty straightforward, honestly.

Like it or not, this has become the canonical way of installing infrastructural software onto your development machine in the Docker (container) ecosystem. Of course, a competent ops dev would never do this in a production system. But for splash intros on marketing websites, yeah — this strom und drang about how it's incorrigible is getting a bit long in the tooth.

Wake me up when there's a Docker-independent hypervisor engine.

Hi there, not sure what you mean by Docker-independent hypervisor engine? Hyper uses Docker images format in order to ship containers, but is totally independent from Docker engine.

I like how the animation in the header of the website requires two of my CPU cores at full speed.

Also offering piping to bash as first alternative for installing something.

I agree with you on both counts, but still find your comment gratuitously negative.

This looks like a very cool thing, and an important missing piece.

It's "gratuitously negative" to ask me to pipe a script I download from the Internet into bash directly.

They really should, for security reasons, spend the 5 seconds it'd take to separate those into two commands, or come up with a better way to do things (which would probably take more than 5 or 30 seconds).

Reminds me of pip (python packaging manager), who will, almost narcissistically, remind you about how important it is to use secure connections for python repos, and in fact will refuse to run without a bevy of flags set explicitly allowing any kind of insecure or unverified package, but first ask you to download and run a python script with an embedded binary, because that's the only way they could think of to get bootstrapping working.

Oh, believe me I know all about it.

But drive-by potshots at other peoples' impressive work is gratuitously negative. Why not nitpick the copy editing, while we're here?

Pipe to shell grinds my gears. At the very minimum, it should be replaced with `curl -o https://hyper.sh/install && bash install`, to at least validate that curl believes the http request returned OK. Realistically, that's no different from how all software everywhere is installed until/unless it's incorporated into a previously-trusted package manager repository.

"Separate those into two commands"? That's it? Really?

This just screams "cargo cult".

Not really. By separating the two commands and using && between the two, you ensure the script will download successfully before executing, instead of potentially leaving you with the software half-installed.

"When VMs take tens of seconds to boot, Hyper is able to launch instances in sub-second"

tens of a second would be +- same as sub second ;)

Tenths, maybe; tens—not so much.

Reply on news.ycombinator.com