Hacker News Reader

Comment by kazinator | Hacker News Reader

Reasoning is that which knows that it lacks some necessary knowledge, whereas knowledge isn't aware that it lacks some necessary reasoning.

View on HN

Fascinating - makes me think of the distinction between dreaming and being awake. Only in the latter state one can tell the difference.

Not quite true, see lucid dreaming.

To be fair to the GP comment, I've personally never experienced lucid dreaming, so for me the distinction holds - when I'm dreaming, I never know I'm dreaming, and mostly I don't even remember my life awake, I only remember whatever's in the context of the dream.

That would be a third, distinct state from each. Dreaming while knowing that you are is quite a thrill!

This has to be one of the most insightful sentences I've ever read.

Would you mind expanding on it a bit? I do sincerely appreciate its pithiness, but curious to read it explained a bit further.

Think of it as: reasoning=computation, knowledge=data. Data alone doesn’t say it must be computed. But computation, by definition, is attempting to create data (the result) that doesn’t exist. Thus: knowledge isn’t aware it must be reasoned about, but reasoning knows it’s trying to find (deduce, compute) knowledge it lacks.

In the world of large language models, what part of "reasoning" is hard-coded and what part, if any, is learnt?

Is reasoning simply a scan/search of your vector space (i.e. your knowledge) according to some hard-coded algo?

Thanks for that!

I disagree. If you have knowledge, and you don't try to do anything other than compress it to make space, reasoning about that knowledge will come about by sheer unintended consequence once the patterns your compressing on reach some threshold of sophistication.

By definition, an optimal compression algo is a dimensionality reduction algo. A dimensionality reduction algo lets you do a bunch of machine learning tasks.

Reply on news.ycombinator.com