Data movement bottlenecks to large-scale model training: Scaling past 1e28 FLOP

It would be really amazing for ai folks if they would crack a book on HPC open every now and then.