Comment by mtlmtlmtlmtl | Hacker News Reader

Comment by mtlmtlmtlmtl | Hacker News Reader

Parent Full thread

"Qualia" is mainly a term used by certain philosophers to insist on consciousness not being explainable by a mechanistic theory. It's not well-defined at all. And neither are "consciousness" and "sentience" anymore, much due to the same philosophers. So I no longer have any idea what to call any of these things. Thanks, philosophy.

I like to think of consciousness as whatever process happens to integrate various disparate sources of information into some cohesive "picture" or experience. That's clearly something that happens, and we can prove that through observing things like how the brain will sync up vision and sound even though sound is always inherently delayed relative to light from the same source. Or take some psychedelics and see the process doing strange things.

Sentience I guess I would call awareness of self or something along those lines.

As to your query, I've certainly met people who seemed incapable of commonplace logic, yet certainly seemed to be just as conscious and sentient as me. And no, I don't believe these language models are sentient. And I doubt their "neural anatomy" is complex enough for the way I imagine consciousness as some sort of global synchronisation between subnets.

But this is all very hand-wavy. Thanks, philosophy. I mean how do we even discuss these things? These terms seemingly have a different meaning to every person I meet. It's just frustrating...

We know regions associated with consciousness in the brain. A recent paper took that knowledge and did something kind of clever - they looked in species where that region was under development. Out of it came a lot of conjecture related to the region being used in the development of complex self-simulation motor control.

It struck me that the intelligence theories of reinforcement learning have been encountering this same paradigm of self-simulation being necessary. A Tesla for example when it has to go down a street that also has another car needs to do a rollout not just for itself but the other car in order to actuate such that it can move in this motor control task. Not being able to model both agents makes it really trick to do planning. These tree search married to modeling problems seems to show up all over the place in the top performing architectures.

For this reason I'm really suspect of the language model view of consciousness; it doesn't seem plausible. Or at least, I'm a little skeptical. In theory for a significantly large enough language model I don't see why you couldn't use a "loop unrolling" argument to posit that for some subset of the network there is an analogous computation to what my best guess about what consciousness is might look like. Worse for me is that when I strong man the argument I run into things like "the training task involves predicting the output of conscious entities and so a truly generalizing agent would be directed to learn just that unrolled loop". I'm not saying I think the language models are conscious, but I do see how reasonable people can arrive at those views.

We probably need to write down a very hard definition of consciousness even if its wrong - or use terms that are better defined - because the ambiguity in the realm of things we don't fully understand makes this an annoying "words" problem rather than a concrete "facts" problem.

"Sentience I guess I would call awareness of self or something along those lines."

Is a self driving car aware of itself? Surely within its logic it represents itself as a vehicle, while also being able to logically represent other vehicles on the road. Its logic accounts for there being similarities (they both obey the same laws of physics and such) and differences (its logic controls its own behavior but not that of other vehicles). So it has a meaningful and useful concept of "self". All of this is required for it to be able to function.

What if some future iteration of a self driving car's vision system allows it to glean useful information from reflections, such as off of store windows? If it correctly can identify a reflection of itself as being itself... well then it has passed the mirror test. But that is a pretty silly thing to assign too much meaning to.

I just don't see "awareness of self" as being meaningful. It doesn't narrow down the definition at all.

I was a little sloppy in my formulation. A better way would be the ability to observe one's own thought processes and integrate them back into consciousness. Meta-cognition is a much better term, I think. It doesn't seem to me like cats for instance have this ability.

You should check out the YouTube of the cat that was trained to use buttons to speak. It sometimes says things that are surprising which it wasn't trained to say directly - like complaining that its owners music is bad. I realize that this isn't contradicting your thesis of language models not being conscious - but I just generally think people should check out things like that cat. So I'm not trying to disagree with you here. I just think the cat is cool.

I'm also a fan of ants. They have agriculture. They farm in their underground cities! They keep livestock too; insect livestock, but still! They give them medicines when they get sick, specially cultivated fungus for example. They are so cool.

I already have, and I'm considering trying it out with my own cat at some point. Cats are way more intelligent than people give them credit for, I agree. But I think you can get very far without the ability to turn the mind's eye inward, and I don't think it's controversial to state that cats don't have a great deal of self-awareness and metacognitive ability.

Neosocial insects like ants are amazing indeed. I often wonder what kind of "mind" an anthill has. Hurts just to think about it.

And, to quote Oliver Sachs: "I'm not sure about octopuses..."

> And, to quote Oliver Sachs: "I'm not sure about octopuses..."

I thought this quote was about the octopus that correctly predicted the outcome of games in a FIFA world cup. But seems it was something else?

...predicted? Or CAUSED???

https://en.wikipedia.org/wiki/Cthulhu

It was Paul the Octopus

https://en.wikipedia.org/wiki/Paul_the_Octopus

Oliver said this during a roundtable discussion on metaphysics with Daniel Dennet, Steve Gould, Rupert Sheldrake, Freeman Dyson, and Stephen Toulmin from 1993.

It's a fascinating discussion with a lot of my favourite mid-late 20th century thinkers. It's up on youtube, lookup any of the names involved with "roundtable discussion". It's over 3 hours long though.

Anyway, Oliver was talking about what kind of mind he imagined various animals having. And that was most of what he had to say on octopuses.

It's a bit of a sad watch too because all the participants except Dennet and Sheldrake are dead now.

A way to make it stop hurting to think about, replaced instead by transcendental beauty, is to abstract pheromones to outcome utility backward induction and the following of pheromones to a single worker being a monte carlo tree search rollout with the pheromones as a guide: they play a mixed strategy proportional to the pheromones. The fading of pheromones becomes something like a weighted moving average. Strong parallels with game theory and reinforcement learning algorithms start to emerge. This model is wrong, but the Earth is also not a perfect sphere.

I find it very fun to think about and have been called wierd for taking about ants before with childlike excitement.

The AntsCanada channel is fun for watching ecosystems interact with each other. So fascinating. :)

Ok, so you are saying a cat isn't sentient?

And yet, most would agree that a cat can feel pain, but a machine can't. I think most people would object if the Google engineer said "I think this AI is every bit as sentient as a cat." They'd still say he's crazy.

I also wonder what would happen if that particular AI actually had logic to "observe its own thought processes and integrate them back into consciousness". Would we then just spin on the semantics, such as by arguing that this isn't really "observing" and it certainly isn't "consciousness"? (how can you define words like "consciousness" without using words or concepts like "sentience", bringing us back to square one?)

So I don't think meta-cognition is the answer either.

Philosophy is currently the only subject which can teach us about sentience (which I would define as the ability to experience.) This is because the subject matter of the sciences is the external world. During the scientific revolution, this limitation of science was, however, more generally and explicitly acknowledged than it is today, where there seems to be a lot of confusion about it. A good book on this stuff is Thomas Nagel’s Mind and Cosmos.

I do agree that the terminology is unfortunately convoluted. The main term is “consciousness,” which does often seem unclear to me.

The basic mystery is just the ability to experience/feel, and is an undeniable part of life, for humans and at least some animals.

Oh, I've tried very hard to understand the views of people like Chalmers, Nagel and Goff. My only take-away is that it's just gatekeeping because science has made more advances on these questions in a few centuries than philosophy has in milennia. They want there to be a hard problem, because then they can keep talking about ill-defined problems all day with no progress in sight, publishing papers. Just happy to be workin'

But science has never said anything about it at all. The subject matter of science is the externally observable and consciousness is an internal category.

Are you just gonna keep repeating that or do you have an actual argument to support this claim?

You’re right. I’m being repetitive. But it’s not really an argument. All I’m doing is saying the definition of science, at least as it’s historically been defined. The only bit of argument I have there is that I think by forgetting this definition we’ve wandered astray.

Reply on news.ycombinator.com