Hacker News Reader

Comment by notamy | Hacker News Reader

> Huge jump from GPT-2 to GPT-3. Huge jump from GPT-3 to GPT-4 … and not so much for GPT-4 (13 months ago) to GPT-4 Turbo (just released). It’s hard not to see this plot as tentative evidence for the hypothesis of diminishing returns. Whatever doubling there might have been, has perhaps come to an end.

Maybe I’m misunderstanding how OpenAI names their models, but was there supposed to be a huge jump from GPT-4 to GPT-4 Turbo? Just from the names it sounds like it’s expected to be in the same realm of model performance, just faster.

View on HN

If OpenAI had GPT-5 they would’ve released that instead of 4-Turbo.

OpenAI was allegedly working on their Arrakis model last year that didn’t end up panning out. It’s clear they’re hitting some sort of limit in the scaling laws, or cost efficiency, or both.

Why should the release GPT-5 if GPT-4 is still at the top of all benchmarks? 95% of the population who are not techies will probably use ChatGPT anyway for everything. I think they are pretty much the leading LLM company in most ways.

It is all financially-driven. All of the AI companies are working on models constantly are are purposely putting out releases so they can be ahead of the game or have some nice news for investors in order to make sure their stocks grow.

OpenAI probably know they are ahead with GPT-5 as well and they are planning to drop it Q3/Q4 after Llama3 (which will maybe trump GPT-4), so they can be ahead again for the EOY earnings.

OpenAI released GPT 3.5 after 4 was done training so that's just wrong.

Your understanding is correct, Gary Marcus is intentionally misrepresenting what GPT-4 turbo is to make a story. It’s clearly the cost optimized version of gpt-4. That’s why it’s much cheaper per token.

I think the point made in the article is that they wanted to build gpt 5 but they renamed it to 4 turbo because the improvements over 4 weren't what they were expecting

Right, but... is there any actual proof of that? I didn’t see any in the article.

From October: https://www.theinformation.com/articles/openai-dropped-work-...

If you read the article, the central claim is that they failed at making a more efficient model, not a larger, better, GPT-5 model.

Then why is cheaper per token, and following the same path as GPT3 with several cheaper turbo versions.

The author suggests that GPT4-Turbo should have been GPT-5, but turned out too weak. Not sure if I buy that personally, I'm quite confident that OpenAI can deliver a lot more than GPT4 does right now, just not in a way that is economically feasible. GPT4-Turbo may be an intermediate release to have something to show while more work is being done behind the scenes.

Reply on news.ycombinator.com