Haha I asked the exact same question at the same time. My own speculation is that it might be convenient if you need to reboot the VM for some reasons?
Or maybe half of the nodes are master nodes only, which avoid running workers on the same node as master, probably a good idea
Hey, I like this idea! Actually, I only have a single node running the control plane. And, to answer the original question, it felt like overkill to dedicate an entire NUC to running just the control plane. So, I decided to run two k8s nodes on each NUC. I guess there's potentially a bit of a performance hit due to the extra overhead of running multiple kubelets, but it hopefully won't matter too much in practice.
How about expanding with 3 SBCs for your master noses? The odroid C4 is a good candidate for this at 50$ each. Or rock pi 4 or PCEngines APU if you want to spend extra on NVMe or SATA storage instead of eMMC.
(Raspberry Pi omitted because, at least from my findings, there’s no attractive USB root storage in SFF so if you want performance and reliability you eventually end up with USB-to-SATA and 2.5” drives, or USB NVME enclosures)