Hacker News Reader

ChatGPT doesn't "grok" anything. It's simply picking the best next word one after the other. Unless your problem statement has statistical significance in its training data, bad results are expected.

View on HN

That’s almost, but not quite true. Or, perhaps, true but too vague. There is a certain amount of inference built into fluent language usage. And the statement, as well as where it, itself goes, contextualizes where it goes next. Those two taken together enable it to do a surprising amount of simple yet powerful inference.

A claim asserted without evidence as usual. Given that accurately predicting the next token in text requires reasoning we should assume that reasoning would be an emergent property of LLMs so long as the architecture is capable of it.

Do you have a reason to believe that an LLM like GPT would be unable to learn how to reason?

The original assertion is literally how LLMs work. Saying that there's reasoning involved is the extraordinary claim here, and the one requiring evidence, not the other way around. Otherwise you're just asking them to prove a negative, which cannot be done.

I don't see how, given current implementation, an LLM can come up with an original thought or invent something novel. Statistics != reasoning.

Reply on news.ycombinator.com