Hacker News Reader

Comment by MuffinFlavored | Hacker News Reader

> Apple realized that 1/OpenAI is no longer the ONLY game in town

Who is the other game in town for them?... Meta? Google?

Anthropic as well

No one from the list (also add Anthropic to that list) is SOTA right now, but maybe Apple wants to keep its options open as the gap closes between OAI and the others.

> No one from the list (also add Anthropic to that list) is SOTA right now

I don't use OpenAI's products. Is there an area where they are clearly ahead? I thought it was at least a tie in all categories.

Last time I checked, o1 did much better in PlanBench that models from other providers indicating better perf in that area [1][2]. Probably that lead will rapidly decay as other players implement similar approaches to "reasoning"/LRMs.

[1] https://www.arxiv.org/abs/2409.13373

[2] https://x.com/omarsar0/status/1838353480672563581

o1, which you seem to be referring to, is at best slightly better at specific tasks than Sonnet 3.5 while being completely unviable economically for production use, despite the amount of subsidizing OpenAI has been doing to make them lose a cool $5 billion this year. GPT has barely improved over the last 1.5 years [1].

[1] https://aider.chat/assets/models-over-time.svg

Yeah, I know. I didn't say it was incredibly better in every bench, but there are reasoning and planning benchmarks it does much better (not just slightly) than Sonnet 3.5 and the rest right now, and that still that makes their offering SOTA vs the others, even if its not in terms of cost.

I put sources in another sibling comment, and as I said in both comments, I think that gap will close. I don't expect OAI to keep their moat much longer, especially with the people they've lost.

PS: I say all of this as a constant critic of OAI: both how they behave and their offerings. But credit where credit is due, I don't see out-of-the-box alternatives to OAI's rlhf'd CoT at any price point right now.

Reply on news.ycombinator.com