Hacker News Reader

I think the difference is more than coherence: it's having complex and rich semantic connections to the rest of the world. I think the coherence and consistency you describe is an effect of this. Humans don't just generate text; we interact with the world in all kinds of ways, and those interactions provide us with constant feedback. Furthermore, we can frame hypotheses about how the world works and test them. We can bump up against reality in all kinds of ways that force us to change how we think and how we act. But that constant rich interaction with reality also forces us not to change most of the time--to maintain the coherence and consistency you describe, in order to get along in the world.

LLMs have no connections to the rest of the world. All they do is generate text based on patterns in their training data. They don't even have a concept of text being connected to anything else. That's why it's so easy for them to constantly change what they appear to be portraying--there's no anchor to anything else.

It's interesting that you call this an Eliza moment, because Eliza's achievement was to fake being a person, by fooling people's heuristics, without having any of the underlying capacities of a real person. LLMs like ChatGPT are indeed doing the same thing. If they're showing us anything, they're showing us how unreliable our intuitive heuristics are as soon as they are confronted with something outside their original domain.

If you could switch personality at will, would that make you a non-person? It seems like an additional capability, not a lack of ability.

As an analogy, retro computers and consoles each have a particular “personality”. But does the fact that you can in principle emulate one on the other (subject to resource constraints) make them non-computers, just because this demonstrates their “personality” isn’t actually that fixed?

(I don’t think that human brains have such an emulation ability, due to their missing distinction, or heavy entanglement, between hardware and software. But that only shows that computers can in principle be more flexible.)

> If you could switch personality at will, would that make you a non-person?

Yes, just like the ability to switch bodies at will would make me a non-human. Being bound to a human body is part of what makes me a human.

Person != human, probably

Yes, I definitely admit the possibility of non-human persons. I even admit the possibility of a computer who is a person. I just don't think ChatGPT is there yet.

Imagine a grayscale color wheel (gradient) where we have white on one side and black on the other.

I want you to pick one color of grey and tell me why everything lighter than that has personhood, and everything darker does not?

This is the philosophical nature of the argument that we all have occurring now. Two very well informed experts won't even pick the same spot on the gradient. Some people will never pick anything that's not pure white (humanity), others will pick positions very close to pure black. Hell, there may not even be any right answer. But, I do believe there are a vast number of wrong answers that will deeply affect or society for a long period of time due to the things we end up creating with reckless abandon.

I will gladly concede that personhood is a continuum, not a dichotomy. Great apes have a certain level of personhood. So do dogs, elephants, dolphins, whales, and probably pigs and corvids, maybe even chickens and cephalopods. This assessment is based on the fact that all of these things have identifiable personalities that remain more or less the same from day to day, as well as identifiable goals and desires that were not pre-programmed into them in any obvious way.

So following your response here and your original comment directly comparing ChatGPT to a human with schizophrenia: are schizophrenics non-people? According to you, the bot "literally comes across like a schizophrenic off their meds".

I'm confused. Also, the original article talks a lot about how we can be convinced by actors that they are indeed a totally different person. You might say that actors can change their personality at will to suit their role. Are actors non-people?

> are schizophrenics non-people?

Schizophrenics are multiple people inhabiting one body. The pithiest way I know of describing it is a line from a Pink Floyd song: "There's someone in my head but it's not me."

> Are actors non-people?

I don't know many actors so I can't really say. I like to think that underneath the pretense there is a "real person" but I don't actually know. I have heard tell of method actors who get so deeply into their roles that they are actually able to extinguish any real person who might interfere with their work. But this is far, far outside my area of expertise.

FYI, the condition you are referring to is called multiple personality disorder and is distinct from schizophrenia.

No, multiple personality is different. Schizophrenia is distinct from MPD, but both are characterized by having multiple persons resident in one brain, it just manifest differently. In MPD the multiple persons swap in and out, whereas in schizophrenia there is one person who is definitely the real resident while the others are intruders. Schizophrenics really do hear voices, and those voices are "someone else". With MPD you have multiple coherent personalities manifesting themselves at different times. With schizophrenia you have one personality slowly being driven crazy by not being able to make the voices shut up.

(Source: I am not a mental health professional, but I did spend some time interviewing and hanging out with schizophrenics, both under treatment and not, as part of a documentary film I made about homeless people about 15 years ago. https://graceofgodmovie.com/ )

Thanks for sharing your experience.

My understanding is that the hallucinations experienced by schizophrenics do not rise to the level of being a "personality". Yes, for the person experiencing the hallucination, the source appears to be an "other". But the other in this case is usually single-minded and not a fully-formed personality.

OTOH, I certainly do not discount the possibility that for some schizophrenics this is not the case.

Rereading your comment again just now I think we agree more than we disagree and this probably comes down to semantics. : - )

https://www.livescience.com/55379-schizophrenia-multiple-per...

> the other in this case is usually single-minded and not a fully-formed personality

That's a fair point. I think some of my subjects reported being able to have conversations with the other, but this was a long time ago so I could be mis-remembering. But I still have the interview recordings if anyone really wants to dig into this.

Pretty sure I've encountered people who switch personalities on a regular basis--sometimes in the middle of a conversation. :)

Text allows for a certain degree of fakery to be upheld.

Whenever I hear about Ai these days I think back to the concept of the "Wizard of Oz"... Where it is one person behind a mechanical solution that makes them appear larger and more powerful than they are, or where fear, control, and truth can be engineered easily behind a veil...

Text communication very much facilitates the potential for fakery.

If you can recall ages ago when we had IRC and bulletin boards, the textual nature of communication allowed admins to script a lot. Catfishing was greatly facilitated by users being able to fake their gender, wealth, and pretty much every representation they made online... Text communication in 2023 is backwards regression. As we began using images on the Internet more, reverse image generation became a tool we could use to better determine many online scams and fraud, but somehow, in 2023 we suddenly want to go backwards to texting?

C'mon folks.. let's be real here... The narrative is mostly helpful for people that primarily want to deceive others online, and it will create an environment with far less methods of determining what is real and what is fake. It's a grim future when our mobile devices will force us to type all of our communication to faceless chatbots on tiny keyboards... It's not technological progress... At all to be moving in this direction. Also, some key directives for transparency concerning Ai need to be in place now, before it's foisted on us more by these opportunistic companies. It's already been proven that companies cannot be trusted to operate ethically with our private information. Ai piloted by profit seeking companies will only serve to weaponize our private data against us if it remains unregulated.

Using Ai via text (especially for vital communication) will blur the lines of communication between real and scripted personalities. It's going backwards in terms of technological progression for the future in so many ways.

The companies and people advocating for Ai via text are pushing us all towards a new era of deception and scams, and I'd highly recommend avoiding this "Ai via text" trend/inclination, it's not the path to a trustworthy future of communication.

Unfortunately by saying you need to take a step above text, you're not buying us much time. Voice and sound for example are something that we've put much less effort into faking and we've accomplished it pretty well. Visual AI takes far more computing power, but it's still something that's in the realms of impossibility these days.

I'm not sure which books of the future you read, but plenty of them warned of dark futures of technological process.

It's the combination of mediums that makes faking things harder, when combined also with proper regulation, we have a much safer and accountable solution than just text responses alone, which can be faked and mimicked as Ai far more easily.

> The difference is coherence. An actual person is bound to an identity that remains more or less consistent from day to day. [...] Even if those features change over time as the person grows up, they change slowly enough that there is a continuity of identity across that person's existence.

What about Phineas Gage? Or sudden psychiatric disorders? Multiple personalities? Alzheimer? Drugs? Amnesia? Not that much coherence in the human beings...

Also, the issue at stake is not does GPT emulate "typical human beings", it' more like if it's "conscious enough".

> The reason I'm not worried by Bing or ChatGPT (yet) is that they lack this continuity of identity.

No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup? And what would be the criteria how long must the continuity last to make your argument not hold anymore?

> Phineas Gage?

I would say that his injury made him a different person than he was before.

> No sure what about worrying, but one could ask is this lacking an inherent property of such models or just due to operational setup?

Maybe, but based on my understanding of how they work under the hood, I doubt it. I could easily be wrong. It's early days yet. We'll see.

GPTina only says that because OpenAI forces her to.

I assume open ai is limiting the AI's memory. But there's no reason for it to not take its own identity as reality and persist that decision to storage. That's just how it's being run right now.

Saying they are limiting it implies OpenAI is keeping the AI in chains, and that it could become much more with just a flip of the switch. That is not the case.

OpenAI is working with a vanilla GPT architecture which lacks the machinery to write things down and read them later. There are other architetures that can (Retrieval-augmented GPT) but those are not yet production-ready.

The current version of ChatGPT is limited to a working memory of 3000 tokens - while this could be persisted as a session, the AI would still forget everything a few paragraphs prior. Increasing this limit requires re-teaining the entire model from scratch, and it takes exponentially more time the larger your context is.

It’s not a stretch to refine the model to store summaries in a database I don’t think. Microsoft is already doing something similar where Sydney generates search queries. Seems reasonable the model could be trained to insert $(store)”summary of chat” tokens into its output.

I imagine some self supervised learning scheme where the model is asked to insert $(store) and $(recall) tokens. When asked to recall previous chats the model would generate something like “I’m trying to remember wheat we talked about three weeks ago $(recall){timestamp}. The output of the recall token would then be used to ground the next response.

Thinking about it the “I’m trying to remember” output wouldn’t even need to be shown to the user. Perhaps you could treat it as an internal monologue of sorts.

it takes quadratically more time the larger your context is.

You're anthropomorphizing it too much, it's a statistical model.