Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so convincingly without resorting to trivial attacks. The burden of proof is on those with extraordinary claims and the claim that AGI is already evinced by any number of models or that current implementations will lead to AGI are very extraordinary indeed.
By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.
I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.
I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any.
People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed:
1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
No you didn't, and that's why I didn't limit my rebuttal to existing models alone.
> 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity.
> 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow.
I think you're getting a bit lost in the debate! Let me start again:
Some people: "Hurr durr. AI is just a stochastic parrot. Humans definitely aren't stochastic parrots. Therefore AI can NEVER be sentient or truly intelligent like a human."
Me: "Woah woah that's quite a bold claim that humans definitely aren't stochastic parrots. Do you have any evidence for this very bold claim?"
Some people: "Mumble mumble creativity... emotion... philosophy. Ok fine, no."
At no point did I claim that there is any reason to believe that human intelligence is a stochastic parrot.
This is the second time you've made accusations, which at this point feels like you're gaslighting me but I'll assume good intent. Here's what you claim:
> At no point did I claim that there is any reason to believe that human intelligence is a stochastic parrot.
Here's what you said:
> 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
> Here's what you said:
> > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
Exactly. I did not say there is reason to believe human intelligence is advanced parroting. I did say there is no reason to believe that it isn't.
Those are two totally different things. Perhaps a simpler example would help:
"I have no reason to believe that you aren't Australian."
"I have reason to believe you are Australian."
See?
And? My thoughts can create a mood, my mood can influence my thoughts. Do you seriously believe that this isn't a trivially easy thing to get even just a LLM to do by itself with the right system prompt, let alone more complex models? Sure, my guess is that it would probably just be role-playing rather than having qualia, but so what?
> The Chinese room argument is arguing intentionality
I agree with IshKebab about the Chinese room. Just because Searle was trying to argue that because the human within the room doesn't know Chinese therefore the system as a whole doesn't, doesn't mean his argument works. I'd say it doesn't prove anything either way, because the human in that room serves the same role as a homunculus in a human brain, leading to an infinite regression problem: https://en.wikipedia.org/wiki/Homunculus_argument
I think that is what the parent commenter points to.
I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.