Hacker News Reader

Comment by tofflos | Hacker News Reader

I played with Weave Ignite the other day which is a Docker-like CLI for Firecracker. Sure there were some rough edges but the overall experience was pretty good. If you are familiar with the Docker CLI you will be able to get some virtual machines up and running very quickly.

Two questions in case someone from Weave tunes into the discussion:

I got the impression that VMs needed an SSH server to be accessible. Is this correct and if so will it be possible to implement something similar to docker exec so that I won't need an SSH server on every VM?

> At the moment ignite and ignited need root privileges on the host to operate due to certain operations (e.g. mount). This will change in the future.

Is there a timetable and could you perhaps elaborate a bit as to why it currently requires root? (I don't know anything about virtual machine internals so this isn't a passive-aggressive question from my side. It's genuine curiosity.)

View on HN

Containers are provided by host kernel cgroups and namespaces, therefore the kernel implements attach (exec) operation which is practically running a new proces (e.g. bash) in a cgroup (container).

Virtual Machines are provided by software or hardware emulation which run separate guest OS with own kernel. There is no standard way for a host to let you run any process and interact with its stdio inside guest OS because the host simply is not aware what you exactly run inside.

The solution is to have an agreed connectivity standard both on the guest and the host. The guest can provide SSH server, telnet server serial terminal, irc bot or some other kind of control capability. Then of course host needs a tooling too, e.g. SSH client.

Are there any real alternatives to SSH and/or sftp? E.g. a mutual TLS authenticated HTTP server...

virtio-vsock

Why not use a virtual terminal? That seems like a pretty standard machine interface to use if the machine is virtualized.

>Is there a timetable and could you perhaps elaborate a bit as to why it currently requires root? (I don't know anything about virtual machine internals so this isn't a passive-aggressive question from my side. It's genuine curiosity.)

Not from Weave, but I might have an idea as I've played with Firecracker a bit. When you start up a Firecracker VM, you need to provide it with a rootfs drive, which is a file containing the root file system to be used for the VM. Ignite uses OCI images, so I guess they are doing something similar to [1] in code, the `mount` part requires sudo, so that would be my guess to why you need root.

[1] https://github.com/firecracker-microvm/firecracker/blob/mast...

I’ve personally never needed to ssh into my “cattle” even when I could (EC2 instances in an autoscaling group, Docker containers run with Fargate) and I haven’t missed it when I couldn’t (lambda). For Fargate/Lambda all console output goes to CloudWatch, for EC2 tasks (legacy Windows), we use Serilog with a CloudWatch sink.

But more generically, if you have to log into your cattle for troubleshooting, you probably need a better logging infrastructure.

Then again, if you are referring to how to initially install software, wouldn’t you usually just create an image for it to run?

It's for initial installation.

I use it to figure out what the image and orchestration settings should be. Packaging an application for containers and container orchestration takes me many, many, attempts to get right. Personally I'm unable to divine the correct combination of settings by reading documentation alone so I try something, enter the container, and look at the outcome.

It's nice to have ptrace once in a while.

The only time I can think of where copious logging wasn’t good enough for “remote debugging” is when I was writing C code trying to figure out why I was killing the call stack or overwriting memory.

Having metrics and logging is a requirement to find out that there is a problem, but it doesn't help much to find out what the problem is.

It depends on the granularity of logging. What can local debugging tell you that copious logging at the “debug” level can’t?

See https://github.com/firecracker-microvm/firecracker-container...

At least crosvm which this is based on offers serial device emulation.

> I got the impression that VMs needed an SSH server to be accessible. Is this correct and if so will it be possible to implement something similar to docker exec so that I won't need an SSH server on every VM?

If you want the capability to exec processes from the host into the VM, I think either Docker API or Kube API is the thing for that, as I understand it. If you could kernel exec processes directly into a VM, then it would not be isolated from the host, this seems almost tautological.

You can arrange for process execution another way than SSH, Docker, or Kube API but regardless of what shape it takes, it will still be an entry point in similar fashion to any of these, as the MicroVM or VM runs its own kernel on KVM and does not talk to the host in this way. Perhaps someone knows more about KVM and can clue me in further if there is more here than meets the eye and maybe what you said is possible.

If you don't need the isolation of a proper VM and were only looking for a roughly VM-shaped system that you CAN "kernel exec" or use nsenter to get processes into, you should look at Footloose[1].

I'm suggesting what you are looking for is actually a container that looks more like a VM or bare-metal machine from the perspective of inits and with the vantage point of the processes running inside.

By default, Footloose nodes are running SSH and SystemD, may appear to work similarly to Ignite VMs, but they are Docker containers that may or may not run privileged mode.

So, if it suits you, then you could start up Footloose "VMs" as I still call them, strip SSH from them, then nsenter or Docker exec into them as you desire, or run Kubernetes on them and use the Kube API including exec.

That is actually a lead in to the next project, known as Firekube[2], kind of a mashup of all these technologies plus one more (wksctl[3]). Firekube integrates both Ignite and Firecracker as well, so you can use it similarly on Linux, (where KVM support is available for Ignite), or MacOS, where Footloose runs container-VMs instead, both behave alike; this suite of projects all put together is a very slick and well integrated package IMHO. It is probably comparable in functionality to Minikube, but with GitOps baked right in.

Disclosure: I am not working for Weaveworks, but we are good friends.

[1]: https://github.com/weaveworks/footloose

[2]: https://github.com/weaveworks/wks-quickstart-firekube/

[3]: https://github.com/weaveworks/wksctl

i think using an emulated serial connection or similar driver interface could be part of a solution to this.

You can replace SSH requirement with virtio-vsock (also supported by Qemu)

https://github.com/firecracker-microvm/firecracker/blob/mast...

Reply on news.ycombinator.com