rubs temples
It's a large language model. It is not smart or dumb. It models the input it is trained on. It is not figuring anything out. It doesn't know anything. It isn't reasoning. It is generating text.
When will the Eliza fever break here?
It's a large language model. It is not smart or dumb. It models the input it is trained on. It is not figuring anything out. It doesn't know anything. It isn't reasoning. It is generating text.
When will the Eliza fever break here?
Super interesting discoveries lately.
Or, OpenAI added a calculator function. Furthermore, I would add:
- ChatGPT hasn't "learned" anything outside of what exists in it's model.
- If it is an AI-based response, it's still derived from token-based inferencing.
- One run of asking ChatGPT something is not enough to prove much of anything.
> I'm sure if someone analyzed the network carefully enough they could probably find the digit add/carry neurons.
They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. They will not find internalized understanding of the concept of math.
No lol. The test doesn't have to be mental arithmetic. and accuracy mistakes creep in at large numbers for multiplication. That's not how calculators work i'm sure you know
>ChatGPT hasn't "learned" anything outside of what exists in it's model.
I'm sorry...and you have?
>If it is an AI-based response, it's still derived from token-based inferencing.
Um...Ok? Lol
>One run of asking ChatGPT something is not enough to prove much of anything.
You can run this on gpt-4 as much as you like. the results are the same. It knows addition.
>They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'. They will not find internalized understanding of the concept of math.
That's...not how that works lol, You don't probe neurons and see tokens. https://clementneo.com/posts/2023/02/11/we-found-an-neuron
Weights don't store the data they train on like that. They are essentially configuration settings.
I just asked my local LLaMA 30Bq4 "What is 23214 + 34243?" and it gave me 57457 so there's probably some innate math abilities in larger LLMs.
That being said, I'm much more impressed that you can literally just tell one of these new LLMs to use a calculator for math calculations (or to do web searches, or whatever) in English and it will understand and actually do so.
That's a lot more impressive (and useful) to me.
Neurons are not tokens. Technical jargon isn't fungible.
>They will not find internalized understanding of the concept of math.
I'm not sure how you figure that. The AlphaGo Zero model is able to learn and reach massively superhuman ability on any board game thrown at, including new ones not in its training set. I don't see how someone or something goes about mopping the floor with grandmasters (and the top champions of every other game) without having some kind of internal understanding of what it is you're playing.
If there isn't an internal understanding then what is it doing? Sure we can say it is just predicting tokens, but obviously it using more than just random chance to predict them or else the output would be gibberish. What is inside those dozens of neural layers may not be a familiar form like logic gates assembled into add and carry circuits, but clearly some type of decision structure exists.
For some philosophy on the matter, here is some Wittgenstein as quoted in https://doi.org/10.1016/j.langsci.2011.04.028
>“Try not to think of understanding as a ‘mental process’ at all – For that is the expression which confuses you… In the sense in which there are processes (including mental processes) which are characteristic of understanding, understanding is not itself a mental process… Thus what I wanted to say was: when he suddenly knew how to go on, when he Understood the principle [of, e.g., the number series to be completed], Then possibly he had a special experience – and if he is asked: ‘What was it? What took place when you suddenly grasped the principle?’ perhaps he will describe it… – but for us it is the circumstances under which he had such an experience that justify him is saying in such a case that he understands." (Wittgenstein, 1968, paragraphs 154–155)
But - I would strongly recommend trying GPT4.
There’s a very good video that is worth watching as well that may not change your mind, but will certainly make you think: https://youtu.be/qbIk7-JPB2c
The thing to remember is that these models are unbelievably huge and deep. We don’t know what is really happening in the layers or what it has really learned.
Thinking that it’s a simple language model that is just predicting the next most likely word is unwise.
All analogies are flawed, some are useful. /Smug
No human can understand what really happens in the billion of calculations done for each token in gpt-4 so how can you claim that there is surely no thought process going on? It can solve some riddles, it can draw pictures and it can reason (to some extend). How is that just generating text to you?
In the end this argument doesn't matter because how it was made is irrelevant. What matters is what it is and can do.
Everyone needs to calm down and realize we don't need to have all the answers right now.
Though if it is it's very different from us, since I can barely recall what happened in the morning.
What might be a couple examples of that?
Right now we know what it is, but people are arguing that it has capability arguably beyond the limit.
It's akin to arguing that humans can survive without oxygen, and then coming up with some alternative definition of oxygen, or surviving to validate the statement.
LLMs are not too different in that regard, but harder to quantify, or maybe due to it being very new.
I think it's interesting to discuss the capabilities of leading-edge language models like GPT-4, because (a) they are already exhibiting the ability to perform a wide variety of useful tasks, and (b) it's clear that there is still a lot of unrealized potential here.
Can you clarify the implications you see here? Are you saying that these LLMs are somehow uninteresting or incapable? That there are limits to what they will be able to accomplish even with further improvements? Or something else?
Maybe I'm being too opinionated but I think we should stop dressing up explanations of large language models in misleading terminology. I'd prefer instead to talk about the actual technology and reason from there.
‘It’s just a bunch of equations’ just isn’t a good argument against this tech.
I mean it is technically a high order Markov chain, or at least every published GPT-N is so far. For example, GPT-3 is a 2048-order Markov chain over tokens.
There may be some confusion because of the difference between the technical definition of Markov chain vs. how low-order Markov chains are usually implemented. Maybe you are used to Markov chains that have explicit transition matrices stored in memory and they are trained by counting the number of times a token appears after every prefix. That does give a Markov chain. But technically Markov chains aren't required to have their transition matrix be explicitly stored in memory, and they aren't required to be trained by simple counting (max likelihood transitions). They can have implicit transition matrices and be trained by gradient descent or whatever and still technically be Markov chains.
I saw Karpathy weigh in on this and he said it's a Markov chain in the same way that a computer is a Markov chain. I guess his point is that if you have a high enough order Markov chain, the intuitions and connotations of Markov chains become less useful, and maybe 2048 order is high enough that it crosses that threshold.
Text generation is GPT-4's function. How it performs its function is another question.
The same is true of any computer program. That doesn't make software in general not useful.