Comment by xani_ | Hacker News Reader

Comment by xani_ | Hacker News Reader

Parent Full thread

They went to trash because containers are more convenient to use and saving few MBs of disk/memory is not what most users care.

The whole idea was pretty much either use custom kernel (which inevitably have way less info on how to debug anything in it), and re-do all of the network and storage plumbing containers already do via the OS they are running one.

OR just very slim linux one which at least people know how to use but STILL is more complexity than "just a blob with some namespaces in it" and STILL requires a bunch of config and data juggling between hypervisor and VM just to share some host files to the guest.

Either way to get to the level of "just a slim layer of code between hypervisor and your code" you need to do a quite a lot of deep plumbing and when anything goes wrong debugging is harder. All to get some perceived security and no better performance than just... running the binary in a container.

It did percolate into "slim containers" idea where the container is just statically compiled binary + few configs and while it does have same problems with debuggability, you can just attach sidecart to it

I guess next big hype will be "VM bUt YoU RuN WebAsSeMbLy In CuStOm KeRnEl"

Virtualization is not just "perceived" security over containerization. From CPU rings on down, it offers dramatically more isolation for security than containerization does.

This isn't about 'what most users care' about either. Most users don't really care about 99% of what container orchestration platforms offer. The providers do absolutely care that malicious users cannot punch out to get a shell on an Azure AKS controller or go digging around inside /proc to figure out what other tenants are doing unless the provider is on top of their configuration and regularly updates to match CVEs.

"most users" will end up using one of the frameworks written by a "big boy" for their stuff, and they'll end up using what's convenient for cloud providers.

The goal of microvms is ultimately to remove everything you're talking about from the equation. Kata and other microvm frameworks aim to be basically jsut another CRI which removes the "deep plumbing" you're talking about. The onus is on them to make this work, but there's an enormous financial payoff, and you'll end up with this whether you think it's worthwhile or not.

in a related vein, most of the distinctions that are being brought up around containers vs vms (pricing, debugability, tooling, overhead) are nothing fundamental at all. they are both executable formats that cut at different layers, and there is really no reason why features of one can't be easily brought to the other.

operating above these abstractions can save us time, but please stop confusing the artifacts of implementation with some kind of fundamental truth. its really hindering our progress.

Bringing the features of one to the other is exactly what microvms means.

How would you compare the security of running in wasmer vs the other two options. I know it is a bit apples and oranges. Just curious if it would be harder to break out of a wasm sandbox, or a vm.

With eBPF there is really not much to argue about in security space.

You can do everything.

New toolset for containers covers pretty much every possible use-case you could even imagine.

The trend will continue in favor of containers and k8s.

Let me know when eBPF can probe into ring-1 hypercalls into a different kernel other than generically watching timing from vm_enter and vm_exit.

Yes, there is a difference between "eBPF can probe what is happening in L0 of the host kernel" and "you can probe what is happening in other kernels in privileged ring-1 calls".

No, this is not what you think it is.

I'm not sure what you mean with regards to eBPF but the difference between a container and a VM is massive with regards to security. Incidentally, my company just published a writeup about Firecracker: https://news.ycombinator.com/item?id=32767784

It is pretty obviously not the case that eBPF means shared-kernel containers are comparably as secure as VMs; there have been recent Linux kernel LPEs that no system call scrubbing BPF code would have caught, without specifically knowing about the bug first.

Running a VM is not necessary more secure than running a container. By definition VM allows to run an untrusted kernel code which makes exploiting curtain bugs more feasible. This is especially so with hardware bugs.

What is fundamentally more secure is running applications inside a VM which is what Amazon is doing. The attacker then has to first exploit the kernel before trying to escape the hypervisor.

Except stuff like Kata Containers exist exactly because containers alone is not enough.

WebAssembly on the cloud is only re-inventing what other bytecode based platforms have been doing for decades, but VC need ideas to invest into apparently.

Reply on news.ycombinator.com