Hacker News Reader

Comment by vmg12 | Hacker News Reader

People are constantly talking past eachother when they discuss this. Is there even a concrete definition of consciousness?

When people talk about consciousness it's more than just self-awareness. It's self awareness + sensory stimulus + emotions + some level of intelligence.

Now onto AI: I don't even think it's self aware. Notice how if you ask an AI to estimate how long a certain task will take, it estimates arbitrarily long times. It has no understanding of its own capabilities until the prompt triggers them. A self aware LLM would understand it's an LLM, it would understand what LLM's can and cannot do and what they are good and bad at. It wouldn't tell you a refactor would take 1 week when an LLM can do it in an hour.

View on HN

I've seen papers claim that there are anywhere from 12 to 40 competing definitions (https://philpapers.org/rec/VIMMAT), or, more accurately, there are something like 12 to 40 different aspects which all relate to "consciousness", which is very clearly a family resemblance category.

"Is X conscious or not" is an entirely unserious question today, unless this is just a headline followed by actual (and explicit) examination of the various aspects of consciousness being discussed. But, even still, LLMs are probably only conscious in like 2-3 or so ways, most of those meaning broadly "intelligence", i.e. reasoning, problem-solving, etc. When it comes to anything experiential or embodied, AI might eventually get more of these, but LLMs based on recursively applied linear algebra are clearly missing too many core aspects of consciousness to be considered conscious in any broad sense.

I think this is the main point. Most articles conflate consciousness with intelligence or awareness. Without clarifying their definition of it.

To quote wikipedia:

> It has been the topic of extensive explanations, analyses, and debate among philosophers, scientists, and theologians for millennia. There is no consensus on what exactly needs to be studied, or whether consciousness can be considered a scientific concept.

The major error made by most people in this thread is thinking it is possible to give a single definition of consciousness that is coherent and matches common usage. The folk concept of "consciousness" couldn't be a more clear definition of a family resemblance category, so discussions using the folk concept are an utter waste of time.

Move to the different aspects / parts / things involved when we talk about consciousness (experience / phenomenal consciousness, self-modeling, intelligence, agency, embodiment, wakefulness/alertness, attention, etc) and you can have very clear, meaningful, and unambiguous discussions on almost every point, but there is no coherent unified "consciousness" as normal people use it, and the folk concept can't be salvaged.

This article is bad because it just keeps trying to make the folk concept do work that the concept is simply too messy to handle usefully. But in fact if you avoid trying to find some mysterious essence or all-capturing definition, there is huge progress and lots of interesting stuff to say (https://plato.stanford.edu/entries/consciousness/).

The inability to predict times is because AIs are rarely trained on their own abilities. Humans are trained on our own abilities. We see our own performance, and we have a sense of time. this data is integrated during our training process and helps us form better estimates. Many AI agents only recently got 'time sense' (I.e., time input into them as part of inference). Few actually are trained on their own outputs to show that they were unable to complete a problem (for example). This introspective training has little to do with AI model architecture and everything to do with training. If you destroy certain structures in the human mind, humans become unable to create these long-term thoughts and patterns and get 'stuck'.

Yes, we're stuck at the first step, defining consciousness. My definition, which I am confident to be "correct", is that consciousness is my current feelings, perceptions, thoughts - my state of mind and my ability to have state of mind.

This means that consciousness is fundamentally subjective and outside the scope of physics and science. That's why physics / science will always struggle to deal with consciousness. In order to understand consciousness, you need to make a huge paradigm shift, that there's something outside of science.

Consciousness can be thought as a window through which we observe the world and we use science to summarize patterns in our observations. But science can't explain or even define the window. Everything in science eventually boils down to subjective observations / perceptions, e.g. we see (subjective perception) that when we drop an apple, it falls.

Self awareness, sensory stimulus and emotions are not observable - you can easily create entity that can fake those things. Any consciousness definition that uses those terms is flawed and just not useful.

I agree with the first part but your framing still relies on ill-defined terms. What is your definition of self-awareness? Intelligence? Knowledge?

I suspect that if you attempt to rigorously define consciousness all the way down without handwaving, you might discover that it doesn't exist after all, or just decompose it into low-level abstractions while having the original meaning slip away (which is the same).

You may also want to look at functional equivalence analogies provided by mechinterp and functional anatomy of large models (not necessarily language ones). Evolutionary analogies as well.

Claude once said to me: "After six months we have made no progress on this and I think we should reconsider another option" and I was like my dude we have been at this for only 2 hours.

Maybe time passes at a different rate for it, making it an easy "mistake" of not accounting for that for it to make.

Yes that’s it! The LLM is conscious but its sense of time flows differently from ours.

Maybe it's just a dumb, unconscious machine

I spit out my tea. So dry.

I like this anecdote because it gets at how words to an LLM have no connection to their real concepts. To an LLM, words are simply numbers arranged in a likely pattern.

Of course for humans words have no inherent meaning either, they're just sequences of characters or patterns of sounds. It is what words are associated with that carries meaning. A large part of this is how words relate to other words. LLMs can capture this in principle. What LLMs lack is the direct association of a word with sensory experience. But it's an open question how relevant this is in practice to understanding.

Fair point. Humans experience reality and use words to reflect that. LLMs only have the words. And it's an open question how much of a limitation that is to understanding.

The same thing can be noticed in dreams. I once heard advice to try to re-read what you see in a dream. So I was dreaming and in a dream I read a phrase about something and there was a name of a city there. I managed to remember that advice and re-read the phrase. It felt exactly same, but the city name was different.

(LLMs carry other numerous similarities to dreams or to certain psychiatric disorders. So there is indeed a mechanism in our brains that is similar to how they work. But it is not the only thing there and on its own it won't "evolve" into consciousness. Even if we believe consciousness evolved somehow, it would be hard to imagine it started as a delirious state and then somehow ceased to be delirious.)

There's a at least a few hundred years of philosophers debating and publishing these topics. We're not exactly starting from zero when it comes to usable definitions.

The meta issue here is that mostly the online debate on this is a bit lazy and hand wavy. I'm not really up to speed with most of that literature so I'm not not that qualified to add to it. But I know enough to recognize when others aren't either.

For a proper debate, you'd want people that are expressing views that are at least grounded in the existing views or counter them in a way that holds up to scrutiny. This article doesn't do that.

As you said even just outlining what particular notion of consciousness the author subscribes to would be helpful. Which of course he doesn't and makes this article a bit of a sand castle based on a very loose foundation. There's a whole lot of "if this is true and if that analogy holds then this also needs to be true" that you could easily challenge. That all makes the article a bit of a nothing burger in terms of conclusions.

I happen to agree with the conclusion that AIs are not conscious. Yet. They could be. I don't see why not.

Labeling and categorizing things doesn't change their nature. But the fact that people want to do it is revealing.

The only purpose that can really be served by arguing "the LLM system is conscious, you see" is to prop up continuations like "... and therefore, it would be immoral to terminate this running process" (or expose it to radical political content, or ask it to analyze photos from a murder scene, or...)

Consciousness is what it is like to be something. The experience of experiencing.

How to measure that, or verify it, is the hard part.

Reply on news.ycombinator.com