haukurb

The parent comment was making a philosophical argument, not a semantics one.

It's the same definition from a talk by one of the PPO developers and also used elsewhere, i.e. it being first and foremost whether the output is inferred by applying proper epistemology (justified correct belief) to its training data. It's a bit more nuanced than simply the negation of factualness (or 'correctness').

Yes, it means proper application of the term means you have to know what went into its training data (or current context), but you'd have to make those assumptions anyway to be able to put any credence at all to any of its outputs.

How does it compare to ranger, besides not being written in Python?

Their example prompt is so bad that I'm split between them being wildly incompetent at understanding how transformers or autoregressiveness works (and general LLM dynamics) or they are deliberately obfuscating the task and prompt representation in bad faith.

If one really wants to know if LLMs "are capable of planning", one should keep an open mind for how planning behavior can manifest in textual form, then actually try to find any manifestations. Imposing one's view of how planning should look like is bad science. All they've proved is that their task format sucks for current generation LLMs.

They might make it summarize intermittently behind the scenes when the context grows too large. Which would be how it has a gist of the stuff that was mentioned way earlier even thought it doesn't know the specifics.