Ask HN: ML models trained on decentralized computing?

1 pointfrozencell1 comment

Could models such as Dalle2 (huge amount of parameters), be trained on distributed/decentralized network of volunteer/paid computers in the style of LHC at home [0]

[0] https://en.wikipedia.org/wiki/LHC@home

View on HN

what does backprop look like? how long does a single training iteration take? what happens if someone drops out mid loop? how much does it cost to move the training data around?

Reply on news.ycombinator.com