Ask HN: Phil-of-lang question: What do LLMs mean for indexicals? | Hacker News Reader

Ask HN: Phil-of-lang question: What do LLMs mean for indexicals?

8 points1attice8 comments

In my youth, I was eyeball-deep in analytic philosophy, and I remember that so-called indexicals -- 'you', 'I', etc -- are (or were) philosophically mysterious. This was often explained as the difference between seeing a trail of sugar in the grocery store, and recognizing that your cart was the one with the leaking bag of sugar. (https://plato.stanford.edu/entries/indexicals/)

Like everyone else, I'm agog at ChatGPT, but the part that has me nearly calling up my old professors is, "how, when I tell ChatGPT 'you are writing a screenplay', does it 'know' (in whatever level of scare-quotes are necessary here) that it is writing the screenplay?

This, to me, feels like the most urgent of the many questions ChatGPT raises, but I'm unsure of how even to structure the worry into a proper question, let alone how to answer it. Put another way, I'm less concerned about the putative personhood of ChatGPT (it most certainly is not a person) than I am its possible language agency -- it status as a speaker -- which is a distinct question, and, with regard to language, would (at least to my cursory and obsolete understanding) be less settled.

Any ML-savvy phil nerds on here with a good way of marshalling these worries into something coherent?

ChatGPT knows that it is a ChatGPT because it was programmed to know this. See:

https://scottaaronson.blog/?p=6990#comment-1945915

At the end of the day it's complex mathematical algorithms making predictions about the next word in a sequence of words, based on all the sequences of words it's learned from (snapshot of the web). It turns out that given a sufficient amount of properly labeled data the algorithms can build a model that makes good predictions, thus we finally have a useful chat bot.

While the user experience feels magical, it's helpful to understand what is actually under the hood to put it into perspective.

I'm crap at language. But if you told ChatGPT: "Have Ron write a screenplay" (or variations thereof), would it still output a screenplay or facsimile?

Does this philosophical question exist for hand calculators? They accept input and "personally" respond to it appropriately too.

To personify: What's the speaker-status of a person speaking echolalia?

To religify: What's the speaker-status of the oracle at delphi, or a person overcome by the holy spirit who starts speaking in tongues?

> I was eyeball-deep in analytic philosophy

Then you should be aware (and the rest of your comment implies you are) that

> does it 'know' [...] that it is writing the screenplay?

hangs on a there being a reasonable interpretation of "know" in the case of LLMs.

"Know" is often used as a kind of shorthand in the case of inanimate objects: "the website knew I was using Internet Explorer so it sent me the javascript workarounds", "the thermostat knew the temperature had reached the target so it switched the boiler off" - perhaps shorthand for 'P was causally dominant in machine behaving like so' becoming 'machine behaved like so because it knew that P'.

I think "ChatGPT knows who directed Lost Highway because it answered 'David Lynch' when I asked it", "ChatGPT knows that Hitler is a touchy subject so it refused to write a review of Mein Kampf" and so on are similar in their ascription of 'knowledge' to ChatGPT.

In the same way that the thermostat is engineered to turn off the boiler when the target temperature is reached, ChatGPT is engineered to make linguistic productions that accord with its design goals, including being factually correct on matters of general knowledge and avoiding toxicity, the corpus and training providing the means of achieving these goals.

Taking the above on board, ChatGPT's "knowledge" of indexicals doesn't seem any different from its "knowledge" of David Lynch or Hitler. The statistical relations instilled by its training make it likely to use indexicals in a way that conforms to the conventional human use of them: it 'knows' how to use indexicals.

There's also a different reading of your question: "is ChatGPT programmed specifically to take account of self-knowledge - is this special-cased in code or training?" (which I guess it may be, since it seems informed about itself despite its training corpus dating only up to 2021). But while this may be an interesting programmer question it seems philosophically inert; either way, we're still talking about engineered thermostat knowledge.

Maybe a more interesting question: what distinguishes the knowledge in "I (as a non-LLM human - trust me!) know that David Lynch directed Lost Highway" from the 'inanimate knowledge' exhibited by thermostats, websites, and ChatGPT?

It would seem to me that there's a different class of knowledge at play with knowing e.g. that someone is speaking to you. I mean, Google is pretty good at answering questions you type into the search bar, these days, but it doesn't give me the creepy feeling that ChatGPT gives me, precisely because its answers seem to be based on association and relation. Google does not raise my neck-hairs, and ChatGPT does; I do believe it is because the latter does seem to understand indexicals, whereas the former does not, because when I say 'you' to ChatGPT, it reacts *as though it understood that it was being spoken to*.

Its powers of imagination (e.g. simulating a linux box) are pretty amazing, but it's the eerie feeling that it's playing along because I asked it to that, for me, puts my jaw on the floor.

I guess I'm groping towards a reductive theory of the uncanniness that ChatGPT occasions --- that it's the indexicals.

Please allow me to think further on this matter

I imagine a bootstrap prompt that looks like:

- There is.

- There is a Large Language Model and there is a world.

- ...(magic)...

- *You are the Large Language Model.*

Basically I want to know what the magic looked like that took it from a distinction between two things, and the (apparent, pseudo-) understanding that it understands itself to be one of those two things. Self and other. Do you follow?

Based on the answer at throwaway81523's link is appears to me that the answer is that ChatGPT has a drive to process input, but no means to process input, and the programmer gives ChatGPT a processing template that ChatGPT can use to process input. And that particular processing template says that processing input using this template means that the processor's referent is the ID ChatGPT.

So the "other" is input. While the "self" is processing template. And the actual ChatGPT is a machine-learning database with an impulse to use a pre-loaded processing template on input. The ins and outs of the processing template and the machine-learning database (i.e. the actual code) with respect to driving both output to the user and adaptation of the database and/or template I do not know.

Reply on news.ycombinator.com