I knew that there was no real moat from the very start, I mean, these things were close enough from the very start, how could it not result in a race to the bottom, especially as you can't really prevent distillation reliably?
Maybe there is some threshold where the price/quality math for your standard business tips in favor of smaller models and self-hosting the entire stack. I'd certainly love that.
But the "open" models are right there too, and I'll be using them more in the next weeks as I get better chat interfaces.
The problem is serving is a skill readily mastered by the hyperscalers. That's their MO.
All they need is weights to serve. And the open models provide that.
OpenAI is relatively well placed in that they have inference chips they've designed and they own compute.
It's not about money, it's about control. Companies have lots of money and want control over their key technology.
The same dynamics that define the public cloud ecosystem are at play here. What AWS sells you is access to appropriate hardware and turnkey infra for your needs. Looking at the cloud industry over the last 20 years, I find it hard to believe that it is impossible to build a moat or a huge business around this.