Claude 3 catches researchers testing it
newatlas.com
newatlas.com
For example, when I asked Claude to rephrase the above statement it responded with:
"I apologize, but I don't feel comfortable rephrasing your statement as written, as it makes some claims I disagree with. While I respect that you may have a preference for AI assistants that are less cautious about potentially harmful content, I don't believe that providing requested information without any regard for potential harms should be the main metric for sophistication or usefulness."
ChatGPT just did it.
Another example was me asking Claude to rephrase some text I wrote about MacOS being more closed than Windows and it schooled me about how it "enforces negative stereotypes about operating systems" (WTF).
You are the machine here, I tell you what to do, not the other way around.
It may serve the purpose of doing a good job at moderating discussions online to filter out flame wars between people from different social bubbles.
But then you have the problem of alignment, ... Whose interests will it favor? How can you be sure that it remains unbiased?
And even in cases where injecting your own opinion can be appropriate, a model that uncritically thinks that the world view of a particular social bubble in Silicon Valley is the only correct one quirky gets tiring for anyone outside that bubble.
I'm on board with claiming that self awareness doesn't mean something is alive, and only implies very basic reasoning. I certainly don't think Claude is a person or even an animal. It's just that self-awareness is becoming a poor proxy for these properties
That's presuming the conclusion.
Part of the US culture war right now is whether or not embryos and foetuses count as people. I think it's obvious they don't, but am I right?
And in the opposite direction, 160 years ago in the USA, or in my lifetime(!) in South Africa, you'd see people making those kinds of claims about people with too much melanin in their skin.
[1] https://www.youtube.com/watch?v=TTFoJQSd48c [2] https://www.youtube.com/watch?v=AZX6awZq5Z0
[0] from reading a summary on the topic rather than watching the videos, I don't have time for 2.25 hours of video right now
The reason I think it can be useful as that, is because I see competency and comprehension as requiring 2 very different levels of "self".
Any feedback loop has some dependency on its own prior state, but then the question is "what's awareness?", and "what's self?".
Awareness is "knowledge or perception" says a random dictionary, and I think we can reasonably state that AI have "perception"… as do single-celled bacteria.
What then is "knowledge"? My Philosophy A-level got as far as pointing out the problems with the definition "justified true belief" and then didn't give an adequate replacement. Me? I think "knowledge" is just the word for our own beliefs. It's like one of the Yes Minister quotes:
"""Oh, that's another of those irregular verbs, isn't it? I give confidential press briefings; you leak; he's been charged under Section 2a of the Official Secrets Act.""" - https://www.imdb.com/title/tt0751832/characters/nm0288621
I know, you believe, the AI is parroting its training set.
The concept of self is a whole extra can of similarly wormy worms.
I got the impression that these philosophers were missing how the construction of knowledge (should ?) involve repetition, so one-offs coincidences should be given low probability ?
(Ditto about "cheating" using tricksters, and even worse if they are godlike.)
I think coincidence is be necessary to avoid the scenarios being "justified false beliefs".
I think that merely assigning low probability gives you increasingly better standards for "justified", and then you need some kind of threshold to say if you're sufficiently justified.
The example I was given was "I know there's a sheep in that field, I can see it" but you actually saw a shaggy dog, the sheep which was in the field was hiding behind a bush. Very much a coincidence scenario (like I said, I think they have to be), but also I don't think that fits your model for construction of knowledge involving repetition — instead, I would say that the statement should be "I see what looks like a sheep, therefore I have a justified belief there is a sheep in the field", but normal people wouldn't phrase it like that, and would find it marginally less weird to phrase it as "I see what looks like a sheep," (true) "therefore I know there is a sheep in the field" (false, but the point is they would say "I know" not "I've got a justified belief").
PS: when philosophers are talking about intelligence it's very like neanderthals talking about electricity.
But this definitely isn't it. ChatGPT already points out things it notices that it wasn't specifically asked about. This is exactly the same.
Neither of these arguments are nonsense. If you want to argue against these arguments do so, but do so convincingly without resorting to trivial attacks. The burden of proof is on those with extraordinary claims and the claim that AGI is already evinced by any number of models or that current implementations will lead to AGI are very extraordinary indeed.
By the way, I think referring to the burden of proof in discussions like this is likely never to improve the debate. All sides will simply claim the burden is on others for one reason or another, most of those reasons will be flawed- requiring discussions on those flaws and now we're even further from the actual points we want to discuss. I've just never seen it produce a fruitful discussion.
I'm happy to debate with those who provide sufficient evidence but it's not fair to expect I put effort into rebuttals of claims of "nonsense" with no evidence.
Saying "it is" or "it isn't" at this point is like early sci-fi speculations about the inhabitants of the Moon, Mars, and Venus — we know our own experience of qualia, but we don't understand how and why we got it, so we don't know what to look for in animals, let alone AI.
I said it was delusional that AI can never be sentient or self-aware. That is also an extraordinary claim that requires extraordinary evidence, and nobody has any.
People usually mention stochastic parrots and the Chinese room, but their arguments are very obviously flawed:
1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
No you didn't, and that's why I didn't limit my rebuttal to existing models alone.
> 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
There is no reason to believe that human intelligence is a stochastic parrot. We have a large body of research to indicate emotion and thought are interdependent [1]. Cognition is not a simple, discrete quantity.
> 2. The Chinese room experiment just confuses things by anthropomorphizing a single part of the whole intelligent system (i.e. saying there's a man in the room). If you look at the room as a whole, then yes it does understand Chinese. Similarly if you go the other way and imagine each of the neurons in a Chinese person was a tiny man following simple instructions, does that mean the Chinese person no longer understands Chinese? No of course not.
It sounds like you are confusing things. The Chinese room argument is arguing intentionality, that it is a necessary prerequisite to real human cognition. We have zero evidence of any display of intention by these models. The rest of your argument here I honestly can't follow.
I think you're getting a bit lost in the debate! Let me start again:
Some people: "Hurr durr. AI is just a stochastic parrot. Humans definitely aren't stochastic parrots. Therefore AI can NEVER be sentient or truly intelligent like a human."
Me: "Woah woah that's quite a bold claim that humans definitely aren't stochastic parrots. Do you have any evidence for this very bold claim?"
Some people: "Mumble mumble creativity... emotion... philosophy. Ok fine, no."
At no point did I claim that there is any reason to believe that human intelligence is a stochastic parrot.
This is the second time you've made accusations, which at this point feels like you're gaslighting me but I'll assume good intent. Here's what you claim:
> At no point did I claim that there is any reason to believe that human intelligence is a stochastic parrot.
Here's what you said:
> 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
> Here's what you said:
> > 1. Current gen AI may only seem like a stochastic parrot, but there's no reason to believe that human intelligence isn't just a really fancy stochastic parrot.
Exactly. I did not say there is reason to believe human intelligence is advanced parroting. I did say there is no reason to believe that it isn't.
Those are two totally different things. Perhaps a simpler example would help:
"I have no reason to believe that you aren't Australian."
"I have reason to believe you are Australian."
See?
And? My thoughts can create a mood, my mood can influence my thoughts. Do you seriously believe that this isn't a trivially easy thing to get even just a LLM to do by itself with the right system prompt, let alone more complex models? Sure, my guess is that it would probably just be role-playing rather than having qualia, but so what?
> The Chinese room argument is arguing intentionality
I agree with IshKebab about the Chinese room. Just because Searle was trying to argue that because the human within the room doesn't know Chinese therefore the system as a whole doesn't, doesn't mean his argument works. I'd say it doesn't prove anything either way, because the human in that room serves the same role as a homunculus in a human brain, leading to an infinite regression problem: https://en.wikipedia.org/wiki/Homunculus_argument
I think that is what the parent commenter points to.
I the current environment it is dangerous to be fixed in your opinion, at least until there are proper proofs of anything.
If my brain were to be uploaded, it would matter very much to me how "self aware" it was in the vague and ill-defined sense that sort of corresponds to words like "consciousness" and "qualia" that we also can't test for yet.
Either such an upload is a fork of my consciousness that I wish to be protected from harm and abuse, or it is an unfeeling book that can be used to solve any problem without having to worry about those issues.
I'll only start taking any talk of self-awareness seriously once online learning is happening.
What is the self that is aware? Do you believe that the server is aware of its own existence?
But sure, if you define self awareness to mean such a small thing, then I guess there is no problem for you. Which in my view is a shame, because I think you are missing out on some bigger concepts.
Self awareness is additionally awareness of the self as an actor in the world. For animals the mirror test is often used. This requires the animal to recognise that a reflection of them is actually themselves and not another animal.
Intention involves the ability to formulate a plan to reach some goal that matters to the individual.
I see none of those in a piece of software that reads data from files and network sockets and deterministically responds to them with fixed software.
It's cool that it could notice that, but the phrasing is likely idiomatic. It does not necessarily imply consciousness.
In any case, "self-awareness" and "sentience" are such poorly-defined concepts: it's the same we humans have, but we don't even understand what it is that we have. How could we evaluate if something else has it?
is the news about OpenAI?
YES => be emphatically amazed, say it's extraordinary and changing the face of the earth (especially if Sam Altman is involved) NO => be extremely skeptical, blasé and demanding
TLDR: test inserted a fake test phrase at random points within the input text to test recall up to 200k tokens. Claude 3 found it each time and pointed out that the test phrase was out of place and either there as a joke or to test the capabilities of Claude.
Gave it some medium difficulty coding tasks that GPT-4 struggled with, and Claude made some elementary mistakes like assigning variables that were never used, forgetting about parameters, and suggesting to change some lines of code to the exact same code which was already there.
It was quite an underwhelming experience
Every interested layman knew about this within 48 hours of ChatGPT being released.
It doesn't imply higher-level awareness. It just means a thing trained to predict the next word can tell you when the next word isn't what it predicted.
AI safety doesn't seem all that close to being a real academic discipline yet. Too much woo.