There are quite a few bold claims just in the introductory chapter, but we should not discount the authors work because of it. Those are his opinions.
I personally don’t think LLMs can achieve reasoning, or any kind of innate awareness.
I personally don’t think LLMs can achieve reasoning, or any kind of innate awareness.
The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning?
If that is sufficient for reasoning, is my calculator reasoning when I input 2+2 and it returns 4?
If you were asked to make an HTML button that looks like a watermelon, you would start by considering what a watermelon looks like: a green shell, a red interior, and a circular cross-section. You would then take this information and apply it when writing the CSS for your button.
What word would you use to describe this process?
>The explanation without is that the LLM takes the prompt text, applies a math formula to produce the next token, iterates that function to get a set of tokens, and returns the response. Where is there reasoning?
The "formula" is dependent on prior knowledge about the world, which is used to successfully solve a problem that does not appear in the training data. Why assume that reasoning is somehow impervious to mathematical modelling?
If you want to reduce all reasoning to just math, then LLMs reason. But then saying something like the OP of “we’ve made computers that reason” is not very novel or useful. Under that framework, computers always reasoned.
Does ChatGPT even “solve” this problem? How do you know it’s not in the training data? Example of a tutorial for making a watermelon in css from 2017: https://dev.to/munamohamed94/easy-css-watermelon-slice-anima...
[Edit: Though I guess it's somewhat decent to realize that, given the request for a button that "looks like X", it needs to go find images of X.]
sure you can argue choosing green instead of blue as the button color is a kind of reasoning, but that's too similar to memorized associations to count IMO.
You presume, though, that AGI relies exclusively on LLMs.