Hacker News Reader

Comment by dvt | Hacker News Reader

I genuinely do not understand why anyone would pay OpenAI for this. Running something comparable to Jev is pretty trivial. The whole point of paying for ChatGPT is because OpenAI has a bunch of warehouses that can run a zillion-parameter model.

Running a decision model is way easier and much cheaper. Are they really just trying to capitalize on the hype here? It feels like they really have absolutely zero moat.

View on HN

Why would I run it myself? It's $0.10 per million tokens. Dirt cheap. (Jev is even cheaper.)

You could ask the same question about why anyone would rent a VPS. I can just run my own hardware, it's just a computer!

Buy vs rent is not just about what's possible, it's about what's economic.

Yeah and OAI is twice as expensive as Jev, which is kind of my point. And more expensive than open models, which you don't necessarily have to host yourself. Pure bandwaggoning.

For one, to remove network RTT

For my use case it will cost like $11 a month and we already have OpenaAI keys and accounts with billing in place. I don't want to run my own model infra and I don't want to get permission to set up an account with typesafe.ai

If you're in an enterprise that already has a procurement agreement with OpenAI, this means you don't have to onboard another vendor. Bucket platform strategy.

My opinion is similar, but for a different reason: every use case for decision models that I can think of, I don’t want the model to change in X weeks when the lab decides to “improve it” or “make it safer”.

Depends on the quality of the results. These things are driven by text prompts. If it turns out the OpenAI one returns better quality results than open weight variants they'll be rewarded by the market.

Anyone using a decision model like this is going to have to spin up their own evals - these are far harder to vibe-check than regular text output LLMs.

There isn’t a moat in the sense of self hosting but you need a reason for people who don’t want that to stay on your platform. Customers save time and effort managing payments easier this way. However it’s a race to the bottom price wise.

Going to be all about branding and platform stickiness for OpenAI to make investors and creditors whole.

Existing enterprise contracts? Data retention contracts (some have zero data retention contracts)? Staying with a single provider because it's easier to have everything in one place?

There are probably a lot more reasons.

If you work for a company that has a 3 to 6 month onboarding period for new vendors and a lifetime commitment to maintain a whole bunch of vendor management horseshit for as long as that relationship exists, it makes a ton of sense.

Add in a bunch of model governance and oversight for anything you train yourself and it’s pretty much a slam dunk deal.

Reply on news.ycombinator.com