K0smotron: Running Kubernetes-in-Kubernetes

Metal3 is just a wrapper for Ironic from open-stack.

The two work together to re-image bare metal machines and attach them to clusters as nodes which can then be sliced up into smaller virtaul machine nodes to sell to your end user which they can then use to run docker, kubernets or whatever else they want.

Doing this allows you to have full kernel level isolation on a per-tenant basis which is not possible just using normal containers.

It also allows you to pin workloads to specific CPU cores for latency sensitive tasks as well as pass pci express devices through to individual VMs on the host and other fun things

It's really only useful if your problem is 'i have one really big computer but I want to have lots of small computers instead' but that's pretty much every data center so shrug

ClusterAPI (CAPI) enters the equation in that CAPI requires Virtual Machines or Bare Metal machines as inputs which it then uses to create a kubenrtes cluster. Thus, kubernetes in kubernetes with kernel level isolation on a per-tenant basis