Hacker News Reader

Comment by sebastiennight | Hacker News Reader

I would be very interested in hearing about those "iterative models" you seem so convinced have existed "for years" (so, at least since 2024 / GPT-4o). Do you have any sources?

Extraordinary claims require extraordinary proof. So far the only other people I've seen teach this prompting style or talk about models "correcting their own output" were getting their information from AI-generated, hallucinated LinkedIn and TikTok posts.

If this thing exists - which is not just a LLM outputting content serially, placed inside a harness where itself (or another llm) is prompted to review and also output revisions serially - and if a single model can be prompted to output content and "iterate" or rewind it, and it's been widespread amongst "all reasonable models", surely there will be a flurry of sources you can point me to so I can learn.

View on HN

Well... since o1 (Sept 2024), most models generate a "hidden" thinking trace before the visible answer... surely you have seen this by now: "Wait, that's wrong...", "Ah, now I have the full picture...". The model prunes dead ends when it composes the final answer.

Demanding sources for this is odd. It's literally been a headline feature of every frontier model for two years.

I guess you are "technically correct" that no model can "un-emit" tokens... but I don't think that is what anyone was saying or an interesting point to make.

Edit: see also this recent post, which details another place where revisions can happen, upstream of the reasoning token emission:

https://www.anthropic.com/research/global-workspace

You're talking about "thinking" models which are mostly regular LLMs trained to output a "<thinking></thinking>" token delimiter before the answer that will be shown to the user, and that's a clever use of the Chain of Thought idea. All of which I'm completely familiar with.

GP was saying

> That would be true of non-iterative models that just emit an output from beginning to end.

Which suggests that there are

- "iterative" models

which

- do not output "from beginning to end"

which AFAIK is science-fiction.

don't think it's that deep mate. just sloppy wording on GP's part.

The disagreement here seems to be a confusion between the pure model behavior and the behavior that users experience, which is the model wrapped in a harness. It’s normal and natural now to refer to the whole system as ‘the model’ informally, even though that is technically not accurate. That ship sailed a while ago.

Since the LLM is now designed to run in a harness, it’s really not even wrong any more.

Here’s some of those sources that were apparently hard to find:

https://openai.com/index/openai-o1-system-card/

https://arxiv.org/html/2410.10630v1

https://magazine.sebastianraschka.com/p/understanding-reason...

Reasoning models are mostly regular LLMs trained to output a "<thinking></thinking>" token delimiter before the answer that will be shown to the user, and that's a clever use of the Chain of Thought idea. All of which I'm completely familiar with.