What kind of motherboard can accommodate such a set up? There are not enough PCIe slots on many high end boards.
We tried bunch of setups with Threadrippers and EPYC, at the end settled for the ROMED8-2T which is a monster motherboard.
The advantage of EPYC is that because it's so common, we can find used cheaper ones on ebay. They are a bit slower I believe, but we can deal with that by using Nvidia's DALI and decoding images on the GPU rather than CPU.
We're doing photogrammetry not machine learning, running some blackbox software that scaled best with clock speed so threadrippers were the most efficient option.
I put them in 4U supermicro boxes with a noctua cooler with 2 9000RPM delta fans attached to it with rip ties.
They must scream and push air like hell
Maybe some day I'll use it myself to train my plastic surgery outcome estimation visualization GAN or diffusion model if I can figure out how to fine-tune one.