It's not that simple. You couldn't just use any random GPU in the cloud, you need to connect them using the right kind of interconnect (Mellanox bough by NVidia) which is very expensive.
Other providers (Azure and Oracle) do offer InfiniBand, but that creates a problem with "islands" of compute which can't be easily expanded. There are also more security considerations when using IB in a multi-tenant environment which makes it not such a slam-dunk advantage as you might first assume.