Hacker News Reader

Comment by sebzim4500 | Hacker News Reader

> If OpenAI had GPT-5, they have would shown it.

Not convinced. They had GPT-4 for ages before showing it.

>And the most important thing about the figure is that 4o is not a lot different from Turbo, which is not hugely different from 4.

It's twice as fast and half the cost.

>OpenAI has presumably pivoted to new features precisely because they don’t know how produce the kind of capability advance that the “exponential improvement” would have predicted.

Or they think that users would rather have a faster GPT-4 than a smarter but slower one.

View on HN

> It's twice as fast and half the cost.

There are plenty of usecases that just want the smartest possible AI, and could easily afford 1000x the price before paying a human expert.

How much would you pay a lawyer to write a letter for you? $200 maybe? Well GPT-4o will do it for $0.0025. Would you pay 1000x more ($2.50) for an AI which will shoot you with fewer legal footguns? - of course you would.

Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds.

> Likewise with time, I am totally happy to wait 10 minutes for a better result, rather than having a mediocre result in 10 seconds

theres a massive gulf separating what people say they want and what people want. As well as a massive gulf separating what they will pay and what they say they’ll pay.

while there are plenty of situations where you think you want the smartest possible ai, it’s clear that a pretty smart pretty quick one is good enough for a lot of things.

It could be that using prompt engineering/search techniques like CoT or ToT gives a bigger improvement than scaling up the model.

Exactly. In the long run we want a cheap/fast LLM layer that we can call millions of times in a search/conversation-with-itself pattern. It may be that we'll interact with it the same way, and GPT-6 will be lots of GPT-5's talking to each other transparently to the user, but the path to next-level intelligence needs to get past generating the next token.

Since we don't know the real GPT-4 architecture, it may be doing some of this already.

I think we can be pretty confident GPT-4 isn't doing something like this based on its performance characteristics.

i) The complexity of the prompt/answer does not effect the time per token.

ii) We start getting a response pretty quickly (for short prompts) and then get new tokens at a roughly constant rate

I don't think either of these properties would hold if there were a bunch of models that had to coordinate before they could start writing the output. Certainly techniques like ToT violate them.

expanding to a high revenue but small niche is very challenging, and most importantly it's slow

if some company is going to use a GPT version to do critical work, they'd be insane to jump on it before testing the solution and gaining confidence for a long time, while also evaluating how to exactly incorporate in their workflow without creating total dependency and other such ancillary issues

you would pay 1000x more for the certainty, which you can never have right away - together with the legal liability of the service provider

And not only is it twice as fast at half the cost, while benchmarking better... It also is multimodal over audio, including understanding turn-taking and ad hoc interruptions in human conversation and recognizing multiple speakers. That's a lot of extra capabilities — not every advancement is measured in logical reasoning on text.

Gary Marcus was an advocate of neuro-symbolic approaches to AI, as opposed to deep learning, and he's been incorrectly predicting that "Deep learning is hitting a wall" since the GPT-2 days. His Substack is near-daily low-content negative articles about LLMs, and on Twitter he posts multiple times a day about it and hasn't stopped for years.

> Or they think that users would rather have a faster GPT-4 than a smarter but slower one.

And they are absolutely right.

GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking.

Making it faster enables real-time workflows, and better energy efficiency also gives them much more capability to serve more requests/users while lowering costs. And that knowledge likely carries over to GPT-5 or whatever.

So as the article is suggesting, Gen AI has probably peaked.

I don't understand how you got that from his comment

When you start focusing on increasing speed and trimming the rough edges, the technology is maturing. It can't be maturing AND making huge leaps expected of GPT-5.

even if the tech didn't improve one iota in the next 10 years, the low hanging fruit that remains for the taking in application is just staggering right now

> GPT-4 is already much more than enough for 90% of tasks while maintaining a sane dose of human double-checking.

There's the rub. Does the cost of double checking a 90% solution best the cost of current methods?

Yes. Absolutely.

Who cares if I have to scan through it when it can write 100 lines of rote boring code in 10 seconds when it would take me 5 minutes.

I can type fast but GPT can type thousands of words per minute. I can't compete with that.

Debugging is way harder than getting it right first time though.

The fun thing with all this AI stuff is people are beginning to appreciate the nice thing about computers the whole time was their strict mechanical nature.

My duckduckgo results are starting to have summaries that do not reflect the content of the associated site and contain plausible falsehoods, courtesy of bing, and the content-farming keyword-spamming AI generated SEO slop goes without saying at this point. It'd be very nice if these models weren't also polluting the resources that people use to try and verify things.

> It's twice as fast and half the cost.

is a great trick with language. Because if you just pay for computing time, half the time equals half the cost. But when you phrase it that way, it sounds for some people like 4 times. So ...

For a lot of things you need to pay more for more speed, so that's why I specified both.

Otherwise they could probably make it way faster but also much more expensive by doing less batching, speculative decoding, etc.

Reply on news.ycombinator.com