One interesting thing is if the model can’t fit into GPU memory (sharded across multiple chips) it would be much slower. So one chip would take more than a month.

That’s why these clusters are so valuable, even with Nvidias margins they are still cheaper than using less compute for longer.