Where do you believe humans get their "external goals" from?
> It doesn't matter what you "plug it in to", it will just do what you tell it.
Here's a ChatGPT-4 transcript where I told the LLM it's controlling a human harness: https://chat.openai.com/share/7dbe7fc8-f31c-437b-925b-46e512...
Other than my initial instructions (which all humans receive from other humans!), where did it "do what I told it"? I didn't tell it to open the mailbox.
I also have to point out that even if you could build a ... human harness? (I'm not sure what that is exactly, but I'm sort of guessing) it would be a little mad to expect that ChatGPT could control it simply by saying what it does.
The ability to generate text when prompted is not enough to make an agent capable of autonomously interacting with the world.
And conversely to the spontaneousness of current AI, your body has a constant set of inputs from reality. That is you never stop feeling, hearing, seeing, sensing, etc. Your brain can consciously turn lower the sensitivity on these things (sleeping). Now, if we subject a multimodal AI this continuous stream, how will it behave?
AI is currently compute and power limited. Very little research has gone into continuous powerhungry AI that goes off and does its own thing at this point. And I would counter that it might be really dumb to design such a device without understanding the risks it entails.
Note I said "initial instructions" i.e. all humans are bootstrapped off of other humans, as in:
You are the product of very long line of humans vs. environment, nature & nurture, cultural values, etc. Do you believe the way you generate your next set of "tokens" (thoughts, actions) is completely independent of your "training" as a human? Is your response to a given stimulus completely random?
Can the LLM have a runloop? Can the LLM be situated in a world like you and me are?
If the LLM is just a file on a hard disk in a drawer not connected to anything, then obviously it can't discover novel tokens on its own.
If on the other hand the LLM has a runloop and sensors and basic instructions to do observations and run thought experiments and find new combinations of concepts and name them with tokens, then sure, why wouldn't it be able to?
You might say you define LLMs as "LLMs as they exist today in a human prompt-driven system" but that would be an artificial limitation given the trivial level of programming, even simple bash scripting, that would be necessary to give an LLM a runloop, access to sensors, and basic instructions to discover new stuff.
Perhaps you can, using a tool. However, if we're allowing tools, I bet GPT4 could also write a program that would produce a novel token, by whatever definition you might give.
I don't think GPT4 is AGI. But this is not a good test. (And it does mean something that coming up with a good test is increasingly nontrivial.)
I would have to come up with something no human has ever conceived of. I don't think that is possible, or what point it would make, since nobody would be able to assess the quality of the output in that context?
Is it? The names might possible be original, and maybe the exact flow of ideas, but it's insanely rare for someone to come up with a new concept and not an amalgamation of existing ideas.