Hacker News Reader

In the world of large language models, what part of "reasoning" is hard-coded and what part, if any, is learnt?

Is reasoning simply a scan/search of your vector space (i.e. your knowledge) according to some hard-coded algo?

Thanks for that!

I disagree. If you have knowledge, and you don't try to do anything other than compress it to make space, reasoning about that knowledge will come about by sheer unintended consequence once the patterns your compressing on reach some threshold of sophistication.

By definition, an optimal compression algo is a dimensionality reduction algo. A dimensionality reduction algo lets you do a bunch of machine learning tasks.