Hacker News Reader

We have distributed implementations that allow us to use more than 8 GPUs to train a single network. These machines have 8 cards each, but we aren't limited to a single machine (though having more GPUs connected via PCI than via Ethernet helps)

View on HN

That's interesting. But from what I understand, the problem isn't getting many GPUs to train a single network -- the problem is to get them to scale linearly; i.e. to get each additional GPU to accelerate the process as much as the last. Throwing boxes at neural net training is easy, but people run into serious plateaus scaling GPUs.

Any particular reason you don't have Infiniband on these for interconnects?

Having this chassis with Infiniband and a local disk would be a dream if the manufacturing cost is right as we scale up in our local datacenter where I'm at.

Reply on news.ycombinator.com