Dood, it's not "deciding" to do anything. It's autocompleting commands that statistically follow other commands. It might do anything.
These models don't have will which is why it can't decide anything.
You do this! When you're speaking or writing you're going one word at a time just like the model, and just like the model your attention is moving between different things to form the thoughts. When you're writing you need at least a semi-complete thought in order for you figure out what word you should write next. The fact that it generates one word at a time is a red herring as to what's really going on.
Also, what are these “thoughts” you have when writing and what’s a “complete” vs. “semi-complete” one? Stupid question, yes, but you again vastly over-trivialize the real dynamic interplay between figuring out what word(s) to write and figuring out what precisely you’re actually trying to say at all. Writing really is another form of thinking for us.
A language is a vocabulary and a set of rules which can be described and codified, and is generally learned from others and other prior examples.
But all that describes is a tool called a language. The tool itself is mechanistic. The wielder may or may not be. The tool and the wielder are two different things.
A piece of paper with text on it is not a writer. A machine that writes text onto paper is still not a writer even though a human writer performs the same physical part of the process as the machine.
Similarly, just like it's possible for an mp3 player to make the same sounds we use to express concepts and convey understanding, without itself having any concepts or understanding, it is also possible for an intellect to perform mechanistic acts that don't require an intellect.
Sure, actually quite a lot of human activity can be described by simple rules that a machine can replicate. So what?
You can move a brick from one place to another. And so can a cart. And so can gravity. Therefor you are no different from a field?
It's utterly silly to be mystified by this. It's challenging the other ape in mirror for looking at you.
As we increase parameter sizes and increment on the architecture, they’re just going to get better as statistical models. If we decide to assign terms reserved for human beings, that’s more a reflection on the individual. Also they certainly can decide to do things, but so can my switch statements.
I’m going to admit that I get a little unnerved when people say these statistical models have “actual intelligence”. The counter is always along the lines of “if we can’t tell the difference, what is the difference?”
Draw the line wherever you like, if you want to say that intelligence can't be meaningfully separated from stuff like self-awareness, memory, self-learning, self-consistency then that's fine. But is intelligence and reason really so special that you have to be a full being to exhibit it?
My comment is predicated on the belief that yes, at this moment it is more rational to assume we have a special spark. Moreso, it’s irrational that individuals believe that in these models there’s a uniqueness beyond a few emergent properties. It’s a critique on the individuals, not the systems. I worry many of us are a few Altman statements short of having a Blake Lemoine break.
To look at our statistical models and say they exhibit “actual intelligence” concerns me that individuals are losing groundness with what we have in front of us.
What's impressive is how much it can do by just doing that, because that function is so complicated. But it clearly has limits that aren't related to the scope of the training data, as is demonstrated to me daily by getting into a circular argument with ChatGPT about something.
I swear to God we could have AGI+robotics capable of doing anything a human can do (but better) and we'll still - as a species - have multi-hour podcasts pondering and mostly concluding "yea, impressive, but they're not really intelligent. That's not what intelligence actually is."
However, when people talk like this, it does make one wonder if the opposite isn't true. No AI has done more than what an mp3 player does, but apparently there are people who hear an mp3 player say "Hello neighbor!" and actually believe that it greeted them.
Otherwise I do not know what definition of intelligence you are using. For me I just use wiki's: "Intelligence can be described as the ability to perceive or infer information; and to retain it as knowledge to be applied to adaptive behaviors within an environment or context."
Nothing in that definition disallows a system from being "pure mechanism" and also being intelligent. An mp3 player isn't intelligent because - unlike AI - it isn't taking in information to retain it as knowledge to be applied to adaptive behaviors within an environment or context.
An mp3 player that learns my preferences over time and knows exactly when to say hi and how based on contextual cues is displaying intelligence. But I would be mistaken for thinking that it doing so means it is also conscious.
Well it's no crime to be so handicapped, but if it were me, that would not be something I went around advertizing.
Or you're not so challenged. You did know how to parse that perfectly common phrase, because you are not in fact a moron. Ok, then that means you are disingenuous instead, attempting to make a point that you know doesn't exist.
I mean I'm not claiming either one, you can choose.
Perhaps you should ask ChatGPT which one you should claim to be, hapless innocent idiot, or formidable intellect ass.
The linked article doesn't give us the full transcript of what transpired, so we can't pour over it and analyze what it did, but it went in and messed about with grub, which, used to be, you'd have a cavalier junior sysadmin would go in and do. Now we can have an LLM go do that at the cost of billions of dollars. but the thing of it is, it didn't go in and just start doing random things, like rm -rf / , or composing a poem, or getting stuck in vi. it tried (but failed) to do something specific, beyond what the human driving it asked it to do, but with something resembling intention.
whether LLMs can reason depends on your definition of reason, but intention is a new one.
However i won’t because despite it not being in my training data, i recognize that blindly running updates could fuck my day up. This process isn’t a statistical model expressed through language, this is me making judgement calls based on what I know and more importantly, don’t know.