Wow that is some serious horsepower.

I guess I just don't get why you'd have such small k8s nodes in a cluster, I have a cluster about half that size (in node and core count) but just run k8s directly on the metal. It means that things are quite elastic with an HPA + VPA on things that need it. I feel like we get better overall utilization of the hardware, but I'd be interested in hearing about some of the reasons for your approach.

Off the top of my head, there are probably a couple of 'pros' to doing it your way; one: experimentation is probably easier. Two, I'm willing to bet networking isn't anything too fancy (getting k8s to play nice on bare metal networks is a shitty thing, especially once multiple NICs are involved -- a LOT of networking stacks, ok, all of them, just 'assume' that the first network card it sees is THE network card it should use... and don't even bother to 'sort' them so its the same order every reboot).