Hacker News Reader

Comment by zavertnik | Hacker News Reader

And here I was in bliss with the 32k context increase 3 days ago. 128k context? Absolutely insane. It feels like now the bottle neck in GPT workflows is no longer GPT, but instead its the wallet!

Such an amazing time to be alive.

View on HN

> 128k context? Absolutely insane

128k context is great and all, but how effective are the middle 100,000 tokens? LLMs are known to struggle with remembering stuff that isn't at the start or end of the input. Known as the Lost Middle

https://arxiv.org/abs/2307.03172

In fairness, humans have much the same problem.

sama said they improved it

We can't just take his word for it. This needs experimental verification. It's likely not even close to solved.

people now have access and can do any verification they want

For GPT-4 Turbo, not GPT-4.

Yes, nowhere in the text today was there any assertion that Turbo produces (eg) source code at the same level of coherence and consistently high quality as GPT4.

Altman did specifically say it’s a “better model” than GPT4, but that’s “better” is vague enough that it might not actually be in terms of accuracy.

They specifically said it's a better model

Was there an assertion that it doesn't?

GPT-4-Turbo seems to be replacing GPT-4 (non-turbo); the GPT-4 (non-turbo) model is marked as "Legacy" in the model list.

EDIT: the above is corrected, it previously erroneously said the non-turbo model was marked as "deprecated", which is a different thing.

now with the prices reduced so much even the wallet might not be the bottle neck anymore

Comment will not age well.

It’s insane because it makes no sense. When you read a book you don’t remember the last 100,000 words. It’s so wildly inefficient to do it that way.

Huh? By the time you finish reading a book you've forgotten the book?

The specific words, yes.

Reply on news.ycombinator.com