In terms of terraflops, AWS likely has the largest GPU compute capacity out there, and they could use idle GPU capacity to train their models, and Anthorpic has some top tier researchers building it. Give it time.
Other providers (Azure and Oracle) do offer InfiniBand, but that creates a problem with "islands" of compute which can't be easily expanded. There are also more security considerations when using IB in a multi-tenant environment which makes it not such a slam-dunk advantage as you might first assume.