Hacker News Reader

Comment by a-ve | Hacker News Reader

To the container wizards: Is it possible to orchestrate lxc containers using kubernetes? I've been looking at lxc containers for a while and really would not like to run Docker as root.

View on HN

Yes it is possible, but not using Kubernetes AFAIK. HashiCorp Nomad [1] has an LXC driver which allows you to orchestrate LXC containers.

[1]: https://www.nomadproject.io/docs/drivers/external/lxc.html

Someone is working on it, but its still early days: https://github.com/automaticserver/lxe

LXD has orchestration support natively, though it's not at all like Kubernetes (you are manually moving containers around and so on).

I have heard that some folks have looked into using LXC under Kubernetes (and theoretically the OCI templates for LXC could possibly make this somewhat work) but there isn't an obvious way to do that today AFAIK. And I'm not convinced (given CNI which touches some deep bits of runc's particular behaviour) it would work with everything you'd want it to.

> given CNI which touches some deep bits of runc's particular behaviour

Could you explain? I’m a maintainer of CNI and I know almost nothing about runc, so I’m not clear where they touch.

The first CNI implementation came out of rkt, pre-dating runc by a year or two.

I mean that it likely makes certain assumptions about how runc sets up containers which aren't true of LXC. I'm sure you could get CNI to work with LXC (in fact, someone might've already done that -- I'm not sure tbh) but it wouldn't be something you'd be able to drop-in without at least a bit of extra work. For instance, LXC's hooks run in different contexts to OCI hooks (though we recently discovered that runc runs hooks in different contexts to the OCI spec, but that's another topic).

And historically, yes it might predate the runc binary but the libcontainer code that is now part of runc predated CNI -- and all of our fun idiosyncrasies are in libcontainer.

> likely makes certain assumptions about how runc sets up containers

Nope. CNI takes as parameters a “container ID” (any string) and a network namespace path. No knowledge is needed or implied about how those things fit with actual containers.

There are some circumstances where that might not work as seamlessly as possible (such as with user namespaces -- runc and LXC set those up slightly differently and run hooks in slightly different orders and contexts) but it wouldn't be too hard to get it to work. I did a quick search, and it turns out that making CNI work under LXC did require some patches (though I'm not sure if they were CNI or LXC patches -- the blog post doesn't link to either) but they were all merged a bit less than 2 years ago[1].

[1]: https://s3hh.wordpress.com/2017/10/19/cni-for-lxc/

Reply on news.ycombinator.com