And who funds those providers? They can clearly pull the plug any time. The so called many providers is the same group under the hood.
Nvidia was noted as having most of their revenue heavily tilted against only a few major customers for example.
Nvidia was noted as having most of their revenue heavily tilted against only a few major customers for example.
To be clear: The models are open weights everyone can simply download, because many labs publish them as such. The providers in question are generic hosters. It's the same as if you would get some managed wordpress hosting somewhere.
Start with where the GPU and rest of it comes from.
i.e. whether you go for "only inference" or "both" it'll work out similar somewhere.
Kind of why Nvidia see themselves forced to make such deals, to lock labs in before they loose another to objectively superior inference.
Staying purely with US labs, they just lost Meta and both Anthropic and Gemini models have been available on Vertex without relying on GPUs for a while. OpenAI equally is turning towards Cerebras so yeah, those deals will mean little for inference.
Those circular deals aren’t a thing for the inference providers I tend to go for and they don’t change the fact that using a full GPU purely for inference is like using a pickup truck for a track day. It can be done and the flexibility has advantages, but once you focus on one task, there are better tools for the job.