> The price tag on this stuff, in human capital, data, and hardware, is high enough to preclude that sort of “perfect competition” environment.

I feel like we right now live in that perfect competition environment though. Inference is mostly commoditized, and it’s a race to the bottom for price and latency. I don’t think any of the big providers are making super-normal profit, and are probably discounting inference for access to data/users.