Hacker News Reader

Comment by rmbyrro | Hacker News Reader

> how they can justify the valuations being floated around

I'm sure they have at least a couple jumps in their bag.

Let's not forget inertia, as well. Migrating models is not a trivial project. To gain OpenAI customers, competitors need a considerable jump to justify, which I don't expect any of them to achieve before OpenAI itself delivers their jumps.

View on HN

Products like Cursor literally have a dropdown that lets you switch between models seamlessly. From the surveys I’ve seen, most companies already use more than one model provider. The switching cost is fairly low.

The main problem is actually in making your prompts perform similarly enough with a new model. For sure, switching models is trivial, but it'll disrupt the quality of your service without significant effort in migrating your prompts.

I'm not saying this is an intrinsic moat, just that there's a barrier to change.

To be fair though, you typically end up doing that process even when you remain within the OpenAI ecosystem, because they're putting out new (cheaper) models.

For example, a couple weeks ago I migrated a few game mechanics from GPT-4-Turbo to GPT-4o and it caused significant enough issues that I needed to revert them all until I could go back and retune all the prompts, which ended up taking 2-3 days. That's probably about as much time as it would have taken to tune prompts for any other model, especially since I'm using an API that standardizes format/structure across whatever model I select (as simple as a dropdown) and lets me just focus on prompt messages independent of which LLM they're going to.

This demonstrates a need for more robust engineering practices. Typically you would solve this by testing a validation set against a new model. Some tools like DSPy or Agenta are helping to encourage this. But creating good evaluators for generative responses isn’t easy, and in my experience people tend to punt.

Prompting is different than programming. Auto-testing your prompts is good practice and helps during a migration. But it doesn't change the fact that the prompts won't work the same in a different model.

It's like saying having Python tests eliminates bugs when rewriting a codebase into JavaScript.

That same problem exists everytime your model served from a third-party is optimized/lobotomized. Only and offline model guarantees this wont happen.

With OpenAI's valuation and current interest rates, I don't really understand why they aren't visibly under pressure to show there's a lot of price tolerance, and until they attempt it, no one will see how quickly those model migrations get prioritized and completed.

I think that they can get away with not exploring pricing as much because they’re trying to drive costs down instead and expand the market first. But I agree I’m surprised that prices aren’t being explored more.

Their new mini model is supposedly a lot smaller and cheaper than anything they’ve released before. If it compares to v3 which started this craze, then it’s clearly good enough to capture imagination and drive usage. People have posited that it’s cheap enough for per-click advertising to fund too. OpenAI claims that the lower cost model has doubled demand, which would be a good sign for supply-side growth.

Because people are already leaving for free options found in search engines on one end and on the other many are moving to other providers who offer multi-models or something different.

Reply on news.ycombinator.com