I suspect Netflix are too wedded to AWS (which is weird as Amazon is their biggest threat) but Triton or Red Shift (both of which actually isolate containers using SmartOS and SELinux respectively) make way more sense for other people who want to use the blazing fast IO speed of containers on bare metal.
Not on AWS.
If you are running a webapp and a database with a known workload, not so much.
That one piece of lazy coding held back Javascript on their platform for years, and I'd argue that it as the most significant cause of their browser's market share plummeting (or at last tied with lack of adblockers).
With containers on top of VMs, you now have to manage which containers run on which VMs (and the cloud provider worries about physical boxes), with a pure container solution you just spawn containers (and the cloud provider worries about physical boxes)
They might not be deploying containers on VMs on bare metal because they want to, but because they /have/ to.
I'm not running that combination myself so I wouldn't really know, but I'm not aware of problems with that specific combination or can think of anything obvious.
It might add another 1% overhead for most tasks to run containers on HVM virtual machines.
The one giant exception is network performance. The network is usually virtualized at the VM level, unless you have an "enhanced networking" VM with SR-IOV enabled. For containers it's virtualized a second time.
This makes the combo potentially terrible if you're trying to run high bandwidth stuff on low performance VM's.
I still like the combo because it allows you to give the big ole finger to AWS if they try to lock you in one day. Since your containers are isolated from the VM you can easily spin them up pretty much anywhere else.
- Containers are a combination of namespaces, cgroups, and chroot (maybe). You don't need LXC to use containers. Docker doesn't even use LXC.
- There is no overhead for running processes in containers.
- There is no requirement to virtualize networks for containers. They can be configured to use the host's network directly, at which point you are bound by the host's network capabilities. Otherwise it is typically a combination of bridges and overlay networks for which the benefits outweigh the performance concerns for most workloads.
The overhead for running containers is usually very low but real. The OS needs to partition low level resources that are normally shared and the scheduling introduces some overhead.
I disagree about network performance. The virtualization adds a somewhat small but non-trivial overhead here (the overhead for other stuff could probably be considered trivial)
Here is a paper I dug up on that gives results to back up my ranting. It's a bit old now but probably still holds mostly true. http://domino.research.ibm.com/library/cyberdig.nsf/papers/0...
My point about network virtualization is that it is not required to use linux containers. Yes, some container tools do create network abstractions that add overhead, but they aren't required and most tools allow you to optionally bypass the abstraction and sit directly on the host's network stack.
But I don't see how that would be any worse on EC2 compared to bare metal or any other hypervisor/provider.
Maybe I'm just interpreting too much into the OPs wording and he did not mean that it's a specific EC2 issue.
Now, Netflix, being a priority customer, may get higher limits and such. But average joe public cloud user should keep that in mind before trying to use EC2 for running containers.
Similarly, do you have any data showing that a container system has such incredible overhead compared to the actual application workload? I mean, if that was true you'd think the entire Kubernetes team would be staying up nights figuring out how to reduce overhead.
Tech has too many edgy code names.
See https://blog.openshift.com/securing-dockers-future-with-seli...
But people keep selling SELinux or AppArmor as a solution for multi-tenant container environments, which is just plain false.
The real solution are efforts like like Intel's Clear Containers and Hyper's runV.
Doing a little research: GKE runs on https://cloud.google.com/container-optimized-os/docs/ which is designed for GCE, which runs on KVM: https://cloud.google.com/compute/docs/faq
So it looks like just Triton and Red Shift.
Disclaimer: I used to work at Red Hat, which means I like the people behind Red Shift, but that also means I hate the people behind Triton as part of the early 2000s Linux/Solaris wars.
I'm 99% sure that Google runs one VM per container because that's the only way to make it safe.
Anything else would be insane.
Hypervisors are much, much harder to break out of than a Linux container.
I'm 100% sure you are wrong. You might as well just use VMs.
Containers are not only about safety, you know.