One underlying claim here is that latency within the data center will start to matter more and more for future AI compute.

That is relevant for musks in-space computation. There, latency between compute nodes, due to being hundreds of kilometres apart, cannot be low.

This pretty much limits the size of AI models which can be trained on a space datacenter to a single satellite to avoid incurring huge latency overhead both in training and inference.

With the smartest models getting bigger and bigger, that really doesn't look good for space based compute.