I think with this paper it becomes clear that the adamant denial was just human bias talking. LLMs crossed a certain line here.
You would think people would be amazed or in awe or react in fear at technological break through a and they often are. The weird part for LLMs was the delusion. Many people were in denial that it was any sort of breakthrough.
With those disclaimers, my interactions inform my opinion that the LLM behind chatgpt has no internal world model. It shows no understanding of basic facts and makes very silly mistakes very easily. I have my bias, like anyone else, but in the case of AI in particular, I should say that I don't think there's anything especially magical or sacred about conscience and the human brain (or the brain of other animals, for that matter), and I'm sure it must be possible to arrive at different forms of intelligence starting with the "inanimate" building blocks of hardware and software, but what I've seen so far in LLMs doesn't make me support your statement that they've crossed a certain line. In fact, I'm very much let down by the experience and I'm afraid once the hype goes down, we may be in for another AI winter.
As I said at the start of the comment, I'm well aware I'm not a subject-matter expert. I'm open to being wrong. I just wanted to point out that not all of us unimpressed are in denial. I'd love to be in awe at an AI breakthrough, and I kind of feel I will be in awe sometime during my lifetime, but LLMs are not (yet?) that for me.
Edit: s/are not in denial/are in denial/
Examples of things I'd file as silly mistakes:
- mentioning non-existing settings when asking it about a particular software.
- responding to roughly "give me a way to list all rds clusters that are running a version that will reach EOL within the next 12 months" (which I know is almost impossible to do without scraping, as the EOLs are not returned by the API, but it was worth trying) with a query that compares the version with < 12.0. It did acknowledge the error when I asked "won't that just compare the version number" and, interestingly, it then did add to the response that the API does not provide EOL for versions, but the first response was IMHO bad in a very basic way.
- providing wrong steps in an answer to "give me steps to do X", wrong in a way that would cause the process to go very bad, and then when I point out the problematic step, it responds with a seemingly random alternate step that is also wrong. I guess this part is the one that is less bad, because it loosely reminds me of people who are bad at reasoning and just give back random answers to a logic or math question.
I must say it is surprisingly good (though it will still make up facts at times) when answering questions about elisp though.
Paraphrasing Hofstadter, I'd say if I'm asked about the intelligence of chatgpt using gpt-3 I'd say it's slightly better than a thermostat or a mosquito, but not much more. Furthermore, my (admittedly not very valuable to others) gut feeling is this is not the path to get to what I'd consider an intelligent being, mostly because in my interactions, it makes lots of bad mistakes, and none of the good ones (here I'm using good and bad as qualifiers for mistakes based on my experiences as a parent, and as a mentor to junior peers, where a good mistake is something that lets you know they're on the right path even if they haven't arrived at the right answer/place yet). Try asking it about analogies, for example ("what's a good analogy to explain X to someone new to it"), the results I've gotten are underwhelming, when not just plain wrong.
Edit: s/fall/file/ Formatting.
The difference is extremely dramatic. Any experience with 3 or 3.5 is completely irrelevant for 4.
(I use 3.5 and 4, both in the chatgpt interface and via API.)
- https://lilianweng.github.io/posts/2023-03-15-prompt-enginee...
- https://help.openai.com/en/articles/6654000-best-practices-f...
Not a large percentage of humans would be able to do the tasks listed above without significant experience or training either. Average humans are also not great at reasoning.
I'm in full agreement about humans and reasoning, btw. I just don't think chatgpt (with the version I've used) is anywhere on the same league as the worst (way below-average) humans either. I do think it's quite useful as a writing aid. You do need to review what it gives you, but you'd have to do that even if you were relying on a human writing aid.
You realize Hofstadter is not delusional about LLMs at all. His view point is completely opposite of yours more logical and rational and he doesn't need to use chatGPT 4.
Hofstadter is not a normal person because he is extremely unbiased. chatGPT basically got him to do a 180 on everything he thought and talked about in every book he has written. He flipped, you subscribe to his viewpoints, and you haven't flipped.
Keep in mind he criticized early versions of gpt3 and gpt2. chatGPT changed the game.
https://www.nytimes.com/2023/07/13/opinion/ai-chatgpt-consci...
https://www.lesswrong.com/posts/kAmgdEjq2eYQkB5PP/douglas-ho...
He literally said something along the lines of his core beliefs are collapsing. Paraphrasing. I kid you not. It's abnormal. It's very rare to find a person that can change his core beliefs. Extremely rare. Let's be frank. You are not that person. What will happen is you will reinterpret reality such that it fits your core beliefs and you won't know you're doing this. It's very predictable: Using chatGPT 4 will not revise your opinion on AI because you don't want to face the truth. chatGPT 3.5 is already enough to see how we crossed a certain line here you can't admit it... so your experience will be the same with gpt4.
"Of course, it reinforces the idea that human creativity and so forth come from the brain's hardware. There is nothing else than the brain's hardware, which is neural nets. But one thing that has completely surprised me is that these LLMs and other systems like them are all feed-forward. It's like the firing of the neurons is going only in one direction. And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive.
It also makes me feel that maybe the human mind is not so mysterious and complex and impenetrably complex as I imagined it was when I was writing Gödel, Escher, Bach and writing I Am a Strange Loop. I felt at those times, quite a number of years ago, that as I say, we were very far away from reaching anything computational that could possibly rival us. It was getting more fluid, but I didn't think it was going to happen, you know, within a very short time."
The guy literally flipped bro. He literally just admitted what he wrote in his books are wrong. The books may not be wrong but he believes it now. Degrading hundreds of pages of his own writing and his own core beliefs is a display of incredible rationality, scientific thinking and lack of bias.
I very much respect Hofstadter and the fact that he changed his opinion on GPT (I'm based on what you're saying, I have not read the links you provided yet). I'm not sure that means he changed his core beliefs though, even if he said so. At least, my understanding and interpretation of his work is that we're not the only substrate that can develop "strange loops", and that this is not limited only to organic substrates/living beings. He may have changed his view in GPT, but if he was true to his words, he must have always been open to software being able to develop a being.
I very much think AGI is possible, but:
- I don't think it will happen through LLMs, or at least not through LLMs alone, and
- I don't think it will be human-like. In fact, I expect it will feel very alien to us, and we'll feel very alien to them too. Mostly because we're not brains in jars. We eat, sleep, get laid, feel pain, etc., and all of that feeds back into the loop to make us what we are.
Read what he says. He literally says it's depressing to watch all his core beliefs get dismantled. I'm not making this up. These words came out of his mouth. Follow the links. This is not some minor realization he went through. His entire world view was changed and he's the one claiming this... Not me.
His books which I read are all about recursion and strange loops and he's shocked that LLMs are simply feed forward networks with zero feedback. He claims it debunks everything he wrote.
He has the most to lose in disbelieving what's happening with LLMs. But he's to rational to do what you or most people do. He has to accept the truth and basically admit to being utterly wrong.
Once you read the links you will not agree with him. That's the most likely behavior for most people. But think on this. He has the most to lose and he thinks the books he wrote is wrong while you think the books he wrote is right.
Given the context the conditions are ripe for him to be the most biased and you to be less so, but he believes the opposite against his own self interest. From this you know he's not biased. Most likely you are.
It has nothing to do with fear or dislike of agi. It's just common human bias no one is sure where this lunacy comes from.
1. Did you use GPT-3.5 (free version) or GPT-4 (paid one) to form the judgment? Their performances on harder tasks differ significantly as shown in https://openai.com/research/gpt-4.
2. Have you tried adding "Please think step by step" to some harder requests? This simple phrase gets most current LLMs to perform significantly better. It's a bit like asking students to show their work, which forces them to think more clearly.
Current LLMs, without additional mechanisms, tend to be perform like a drunk or sleepy human, i.e. using mostly intuition or System-1 (as defined in "Thinking Fast and Slow"). The prompt such as "think step by step" asks it to think more in the System-2 style. (There are other techniques which get them to perform even better still.)
I think of current LLMs as a very-well-read, but often sleepy intern who needs strict instructions, feedback, and sometimes extra training if you want them to perform well.
I have tried adding things like "please think step by step" (quite literally that question actually), and also "please make sure you check the facts before answering so you don't include non-existing arguments" (when asking about a cli tool that takes arguments), but I didn't notice a significant improvement.
I like what you say in your last line, though I think a big challenge is that a very-well-read intern got to be very-well-read for at least two reasons: 1) they like to read and can do that a lot (LLMs can do this, not saying they like, but saying training them is analogous to someone reading a lot), 2) they went through a curated list of reading material. My experiences make me think part 2 is a weak part of chatgpt.
I still think LLMs are not the way towards an interesting AI. I don't know why we're insisting so much on natural language. I mean, I can understand this for simpler tasks like a support chatbot, but I wish there was (maybe there is?) good research on building an AI that is not based on human language, since it's one of the worst mediums to communicate with rigor.
This is just the beginning. It is a tiny, tiny view into this new technology.
Nobody with an ounce of intelligence looked at the first iPhone and thought "ah well it's just another phone". It's the potential of this new technology that is so impressive.
How can you not be impressed by this? I am literally incapable of empathizing with your perspective. All I can think is that the world is going to pass you by.
I'm not worried about the world passing me by, I think LLMs and similar technology will increase, not decrease, the need for people who can troubleshoot and fix live production software. I mean I do have my passive income if I'm forced into early retirement, but I'm not fearing that to happen soon.
Mind you, I did say I'd love to be in awe at an AI breakthrough; I'm not a luddite. I just don't see anything in LLMs to impress me, and on the other side, I do see a lot of hype behind it.
More generally, any scholarly article is a claim, and should never be read as automatically true. That's something to keep in mind.
Briefly, a "world model" is a theory possessed by an autonomous agent that describes the entities that exist in the world and how they interact with each other and with the agent, and that the agent can use to make decisions. This is the sense in which "model" is used when people talk about "model based" approaches to AI (such as planning, for instance, which is "the model based approach to autonomous behaviour").
A "language model" on the other hand is a statistical model of the text in a corpus. A statistical model is really a set of events, and their probabilities. The "events" in a statistical language model are usually word collocations, where a collocation is a word A found near a word B, for some measure of "near". If you've heard about "word embeddings", that's a model of word collocations in text corpora.
So the claim that LLMs "have a model" is about LLMs being autonomous agents with a world-model, while "Large Language Model" is about statistical modelling. There's no reason why a statistical language model should have a world model.
Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a representation of the entire world. So where would a language model find it in its training text?
For example if I say "I sat by the sea today", there's no representation of the sea in the word "sea", rather, the representation is inside my head and yours, and the word "sea" is only used to point to that representation [2]. Why would it ever be possible to derive the representation just from the pointer? Like I say, it's really not obvious and I don't think anyone has given a convincing explanation - it's usually words all the way down when people try. In any case, it seems that humans had most of the capabilities conferred by a world model (which we share with other animals) before we had language (which we do not - share with other animals), so it seems likely that a world model must be developed first, before linguistic ability can be built on top of it. But that's a conjecture, of course.
I guess maybe, in principle, it should be possible to treat all of human cognition as some kind of hidden (latent) variable and then train a model to represent it only from its expression in text, but we don't really have any kind of modelling technique with that power. That would be a truly omnipotent model. So powerful, indeed, that it wouldn't need to do that in the first place- it would already be a kind of machine god.
______________
[1] e.g. I believe Tomáš Mikolov, one of the creators of word2vec, has claimed something like that but I'm not sure if I have a source for my own claim; more recently Ilya Sutskever, of OpenAI, has said words to that effect- I have a link somewhere I think, if you are curious.
[2] That's obvious because the word "sea" doesn't mean anything to someone who doesn't know English, but they'll probably have a word for the sea in their language, so they'll have the concept, without needing a specific word for it.
It is pattern recognition with many layers of abstraction. Obviously it will infer semantic relations at some level. It is a type of machine learning. The entire point of machine learning is to generate a model of the data which can be generalized to new inputs.
It would find the world model in semantic relations in text. It would store it in its vast neural network. It would need a world model in order to perform natural language understanding tasks, which is exactly what it was designed to do. When asked about the world it needs a model of the world in order to generate useful answers.
GPT-4 took months to train on a supercomputer and it generated a neural network of hundreds of gigabytes. What exactly was that supercomputer doing for several months and what exactly would the neural network represent if not a world model? Are you thinking it rote learned answers to every possible question?
> so it seems likely that a world model must be developed first
It is "a" model. It can be completely alien to whatever exists in a human mind but still function and exist as some type of model.
> Why would it ever be possible to derive the representation just from the pointer?
It only needs to have a sufficient representation of the Sea that is useful for the tasks we have trained it to do. It doesn't need to derive "the" one true representation that humans have of the Sea, whatever that is.
I believe GPT-4 was trained on a server farm, not a single computer. In any case what it was doing all that time was going over and over the text in its gigantic training corpus, which was of petabyte size, and optimising the objective P(tₖ|tₖ₋ₙ , ..., tₖ₋₁) i.e. the probability of token k given a "sliding window" of the n preceding (or surrounding) tokens.
There is nothing in this objective that needs a world model, and it is really not obvious why optimising this objective should lead to development of a world model, rather than, or in addition to, a model of the training corpus.
It is easy to see how this stuff works. You can train your own language model easily, although of course it would have to be a smaller language model. For example, you can train a Hidden Markov Model on the text of freely available literary works on project Guttenberg, or on wikipedia pages, and without too much compute (an ordinary laptop will do).
In fact, I recommend that as an exercise and as an experiment to gain a better understanding in how language modelling works, for those who are curious about questions regarding their ability to model something beyond text.
A good textbook to begin with statistical language modelling is "Foundations of Statistical Natural Language Processing" by Manning and Schűtze:
https://nlp.stanford.edu/fsnlp/
Or, just as good, "Speech and Language Processing" by Jurafsky and Martin:
https://web.stanford.edu/~jurafsky/slp3/
Or, if you don't have the time for an entire textbook, "Statistical Language Learning" by Eugene Charniak is an excellent, concise introduction to the subject:
Yes a server farm of Nvidia A100 with supercomputing performance.
> It is easy to see how this stuff works. .. you can train a Hidden Markov Model..
No. GPT is not a Markov model.
My questions to you were rhetorical. I wasn't asking you for academic guidance.
But the training corpus describes the world, so a model of the training corpus is a world model
It provides "a" world model and it's a useful model for its intended purpose. Not the "one true model" that is your your own personal life experience since birth.
Of course it cannot derive a representation from just a pointer. Neither humans or machines can do that. The words are not pointers in isolation. They are connected in a network of semantic relations. The model is in the relationships between pointers. Artificial neural network.
Like I say in another comment, this kind of explanation is "words all the way down", but it really doesn't explain where the process ends (or begins) and where the meaning comes into the words.
To give you an example, here is a small semantic network that I just created by hand, that represents is-a and has-a relations:
abc32 is-a c356
7yt4 is-a c356
c356 has-a 902a
c356 has-a 8773
Wikipedia has an article on semantic networks:
https://en.wikipedia.org/wiki/Semantic_network
I haven't read it but from a quick glance it looks informative.
Obviously not. Insufficient data. Try that substitution exercise again with terabytes of data (Common Crawl dataset) then feed it into a neural network and see if it can find any patterns.
As I mentioned in another reply you have a philosophical disagreement and are missing the point.
It provides "a" model of the world. Not necessarily the "one true model" that is your own personal life experience of the world since birth. There is enough information to build "a" world model in all the text ever written by humans about the world. That is obvious.
So your argument is completely subjective and philosophical, not technical.
GPT now includes imagery in its training set BTW.
A "world model" refers to the internal representation of the external environment that an agent (e.g., a human, animal, or machine) maintains and uses to navigate and make decisions within that environment. In other words, it's a mental or computational model of the world as perceived and understood by the agent.
Obviously LLMs have a world model. It's not your favourite model or preferred model or the best model in town, but it is a model of the world whether you like it or not, whether you agree with it or not.
For instance -I'm not trying to be mean and I'm certainly not blaming you in particular, because I've seen this very often- but the reasoning that because LLMs can generate language, and humans can generate language not only LLMs are somehow like humans but also humans are like LLMs is not sound.
For example, walls have ears, cats have ears, therefore walls are like cats and cats are like walls. That doesn't work because walls' ears are not like cats' ears and even if they were, that still wouldn't make walls cats and cats walls, it would just make them both entities with ears.
There is no evidence, basically none whatsoever that general "perfect logical reasoning" is a thing that actually exists in the real world. None.
No animal we've observed does it. Humans certainly don't do it. The only realm this idea actually works is Fiction. and this was not like for a lack of trying. Some of the greatest minds worked on this for decades and some people still don't seem to get it. Logic doesn't scale. They break at real world relationships.
Logic systems are that guy in the stands yelling that he could've made the shot, while he's not even on the field.
Gödel’s Incompleteness Theorems also demonstrates that in any sufficiently powerful mathematical system, there are true statements that cannot be proven within the system. This implies that no matter how refined a logical system you devise, it will invariably be incomplete or inconsistent when grappling with real-world phenomena.
Besides which, you may not hear about them in the news but pretty much all the classical, symbolic- and logic-based approaches of Good, Old-Fashioned AI are still going strong and are doing very well thank you in tasks in which statistical machine learning approaches underperform.
To give a few examples: automated planning and scheduling (used e.g. by NASA in its autonomous guidance systems for its spaceships and rovers), program verification and model checking (the latter has transformed the semiconductor industry and led to several recent Turing awards), SAT-solving and constraint satisfaction (where recent algorithmic advances have made it possible to solve many instances of NP-complete decision problems in polynomial time), adversarial search (AlphaGo and friends aren't going anywhere without Monte Carlo Tree Search), program synthesis (you can generate code with LLMs, but good luck if you want it to work correctly), automated theorem proving, heuristic search, rule learning, etc etc.
To clarify, those are all logic-based approaches that remain the state of the art in classical AI tasks where statistical machine learning has made no progress in the last many decades. You may not read about them in the news and they're not even considered "AI" by many, but that's because they work and work very well, and the "AI Effect" takes hold [1].
Even poor old expert systems are the de facto standard for expressing business logic in the software industry. I guess. Informally, of course.
_________________
Not what I mean. Logic is part of the real world. Logic is not the real world. The idea that you can use this small subset of the world to model the whole thing is what is incredibly suspect. No one has demonstrated this and there is no real reason to believe it can.
>To clarify, those are all logic-based approaches that remain the state of the art in classical AI tasks where statistical machine learning has made no progress in the last many decades
Logic is good at what logic does. Please don't take this to mean me calling logic useless. It's not that statistical machine learning has not made progress. But you won't beat logic on problems with clear definitions and unambiguous axioms. That is very cool but that is clearly not all of reality.
I agree and I don't think there's any kind of logic that can do that, but there is also no other formal system that can, so far. I'm not sure if you are suggesting there is?
>> But you won't beat logic on problems with clear definitions and unambiguous axioms. That is very cool but that is clearly not all of reality.
Certainly not. Logic is a set of powerful formalisms that we can use to solve certain kinds of problem - it's a form of maths, like geometry or calculus. I don't think anyone expects that geometry or calculus is going to solve every problem in existence and the same goes for logic.
No i wasn't. I guess i wasn't very clear in my first reply.
I was mainly getting at this,
>and, consequently, its' a very bad idea to try and make machines that "think like humans", because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers.
No one is scaling up and pouring millions of compute into LLMs for general intelligence because they thought it was an excellent idea before the fact(virtually no one did, even some of the most verbal proponents).
They're doing it because it's seems to be working in a way logic failed to. and logic had the headstart, both in research and public consciousness. Nearly all of fictional ai is an envisioning of the hard symbolic logic general intelligence system that dominated early ai research. Logic was not the underdog here.
The point i was really driving at is that you say "because that way we'll only make machines with none of the advantages of machines and all the disadvantages of computers." almost like it's a choice, like Logic and GPT are both on the field and people are going for the worse player. Logic is not even in consideration because ot couldn't make the cut.
There are two worlds, if you want. For me it's a mistake to try and keep them separated. All the great pioneers of AI were not only this or only that people. e.g. Shannon's MSc thesis gave us boolean logic-based circuits (logic gates) and he also introduced information theory. The people who have made real contributions to AI and to computer science were never one-trick ponies.
An analogy I like to make is that we have both airplanes and helicopters. A flying machine is something so useful to have that we 're going to use any kind we can make. Obviously a helicopter will not compete with a jet for speed, but a jet isn't anywhere as manoeuverable or flexible as a helicopter. So we use both.
>> Logic was not the underdog here.
It wasn't, but there was a bit of a Triassic extinction event, with the last AI winter of the '90s that took the expert systems and basically severed the continuity of logic-based AI research. The story is more complex than that, but logic-based AI was dealt a powerful blow, and progress slowed down. Although again like I say in my other comment, it didn't get completely extinguished. Perhaps, like we recognise birds today as the remaining dinosaurs, we'll recognise the old-new wave of logic-based AI that is hidden by the AI effect.
Nah. Nobody personifies LLMs like this. What you're laying out here is a fundamental mistake that you'd have to be extremely stupid to make. I think barely anyone is making this mistake to even qualify mentioning it.
Seriously who here things that LLMs are anything like humans? That is not the claim. The claim is that LLMs understand you. Intelligence and understanding are clearly orthogonal to "human-like"
This seems like the model found on its own another dimension to segregate data in deeper layers by generalizing and using context when learning. Its cool it can do that, but it still to me seems like drawing a best fit line just with higher dimensions.
I'm sure someone will correct me if they think I am way off.
Could you please explain what the above has to do with a Large Languange Model?
What I'd like to know is why should anyone care about this preprint? Nothing has been stated on how this software will improve the life of the average human being.
LK-99 at least would have had lead.somewhere if it worked.
The LLMs simply activated when locations or times were mentioned, with no understanding of the relationship between different places or different times beyond the fact that different places and times had different tags. Or in other words, no internal model of the world.
The paper has nothing to do with the common sense problem.
While it is a preprint, I do wonder how useful this will be to improve generalization.
Termination analysis on linear polynomials is undecidable over the naturals without many constraints and is undecidable ofer the rationals.
If you think about how sstrees or even rtrees work, this isn't surprising to me that space and time are useful for compression which is required for learnability.
But I think people also see 'linear' and think that means easy. When it can mean serious restrictions on how it can be used.
Hard to explain the connection I have in mind without serious math but here is a paper that may cause some issues with trying to use these internal structures outside of the learnability implications.
It already becomes clear after playing with ChatGPT-4 for five minutes. But a lot of people still refuse to accept it.