> You might want to do some research into the words “science” and “pedantry”, since the former is just meticulous application of the latter. Calling a scientist pedantic is a compliment.
Not only am I extremely well versed in the definition and philosophy of the word science (likely much more well-versed than you), but you are completely and utterly wrong about the compliment part. A scientist is a human, if I call a scientist pedantic during a normal discussion then the scientist will take it as an insult. Do you think a scientist has conditioned his mind into a sort of emotionless robot who can needlessly branch off onto a debate about the definition of the word "science" and "pedantry" when the topic is actuality "machine learning"? No. A scientist can both be pedantic and stupid, being a scientist does not preclude one from being human.
>That aside, you’ve been arguing that these models understand things and citing these papers as evidence. They are not. They are evidence that the ability of these models to generate text based off of their existing training set can easily be finetuned in a number of ways to add training sets after the initial zero-shot learning.
I posted two papers. You're conveniently ignoring the first and naively mistaken about the second.
Part of "understanding" is the ability to formulate new theorems from previously known facts, in order to do this one must "understand" how these facts compose to form new statements. This is what's happening in the fine tuning. It is a demonstration of understanding... that it knows how disparate knowledge composes to form new knowledge. The very definition of understanding.
>That’s all these models do, they generate text based upon some training set. If we define understanding as the ability to extrapolate beyond what one has been told, they are expressly not doing that. Your papers explain this quite well.
Of course. You cannot extrapolate anything beyond what you Observe as well. Can you literally form new knowledge out of thin air? No. You have three things: Existing knowledge, knowledge through observation, and knowledge through composition of existing knowledge.
Without introducing new knowledge, LLMs can be coerced to compose existing knowledge to form new knowledge. Additionally, in the ICL step they can be introduced to new knowledge and form additional compositions there. This has been demonstrated repeatedly.
>Edit: to see more concrete examples of this, look into the unfortunately named “hallucination” ability of LLMs. Once you realize that they only know what they were told and are unable to logically extrapolate the point becomes clearer. I hope that helps.
It's obvious chatGPT makes stuff up. Every one who has worked with LLMs in depth is fully aware of this. It's an obvious thing, you don't even have to "look it up" everyone knows about it.
This claim is made DESPITE the fact that LLMs hallucinate. It's obvious these models are imperfect and it's obvious they have huge deficiencies. But when it doesn't hallucinate, when the answer is Novel, creative, correct and unmistakably not existing in any training set, then we know the model understood the query you gave it.