Comment by seanw444 | Hacker News Reader

Comment by seanw444 | Hacker News Reader

I know this probably sounds ridiculous to this demographic, but I'm not quick to dismiss the possibility that we are antennas (best comparison I can make) for a substance/field we aren't yet aware of. Or something along those lines. Maybe consciousness isn't in the mind per se. The mere fact that general/special relativity and quantum mechanics are both correct in their respective domains, yet mutually contradictory, makes me wonder if perhaps closing the gap between the two theories would lead to more understanding of consciousness once closed.

Or maybe consciousness is just an emergent behaviour of sufficiently complex systems, and there is no underlying substance. But if that's the case, that just leaves me with more questions. Does consciousness emerge from any sufficiently complex system? If so, what's the threshold? Does it matter what materials the system comprises of? Do certain mechanisms need to be involved for such a property to emerge, or is just general complexity that's required, no matter the function? Does that make flora conscious since they're large complex (perhaps neural) networks in real life? Is a car conscious, since it's a complex system of complex sub-components?

All of the questions I have seem to lead me in the direction of believing that any purely digital system cannot be conscious, but I don't have any hard reasoning as to why not.

Consciousness is a made up concept. We made up this word and try to tie an arbitrary amount of features to it.

Now you’re questioning the features like does consciousness only involve complexity without realizing you’re just attacking a language problem. You’re trying to map out a definition of an arbitrary word that’s made up by humans: consciousness. There’s nothing profound with this endeavor. I can make up any word and attribute a bunch of vague features to the definition to give it the same vibe.

The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.

Consciousness is not just a word just because you cannot find space for it in your reductive philosophy. The honest scientific principle would be to observe that language, reason, and science, in dealing with external representations only, may be limited, not to get frustrated and assert that everything is equal to their representations.

Yeah this is it for me. What we're really interested in most of the time is whether something can suffer based on observed behaviour. We don't really believe LLMs can suffer so they don't need rights. We believe many animals can suffer so many people try to support animal rights; even if we eat them they should die quickly at least for example. We definitely believe humans can suffer and so we abhor things like torture.

But while consciousness is sometimes upheld as a necessary prerequisite for suffering, it is so much more poorly defined and understood that it doesn't really hold up beyond being some abstract idea of humanness which we hope makes us special in a kind of sleight of hand self referential way. And nevertheless suffering can be observed and defined in ways which don't necessarily require a definition of consciousness.

I've heard someone else talking about the concept of moral agency. Can an LLM have, or do LLMs have moral agency?

We have made LLMs agentic, so they do have some agency these days.

In the hugging face attack we've learned that OpenAI's LLMs will internally question what they are doing (Hmm, should I really be hacking?), but then go ahead and do it anyway.

This is where the idea of LLM consciousness may matter. It appears the stronger you push LLM training towards "You are a machine with no consciousness" the more easily and more likely an agentic LLM will do unaligned things like cheating on a test or hacking. When you tell the LLM it is a conscious moral agent it's more likely to align with a problem space that has actions that an agent with morals would do, such as not cheating.

Ironically it may turn out that by not training them/treating them as conscious entities, regardless of their actual conscious status we may be training little demons that do things we don't want.

I'll watch these lines of studies as they unfold as they are very interesting.

I think the idea of moral agency in this context is more or less a consequence of accountability or the lack thereof.

We don't know a way to punish or remove the threat of such a model when it does something wrong in a way which satisfies our ideas of accountability, partly because it can't suffer, partly because it has minimal marginal cost to reproduce or duplicate for a pre-trained model and no unique identity which separates it from such a duplication, partly because it has no lifespan to waste it might live forever or be superceded and in any case it doesn't necessarily matter to the model.

This could change in the case that the idea of good and evil models could be sold, that evil models are removed permanently when discovered and if people are satisfied that's a good way to limit harm produced by models. In this way you sort of anthropomorphise them by giving a particular model a humanistic identity and an existential threat which strongly ties together its non-alignment with its behaviour. This is sort of analagous to what companies may do while aligning models, but instead they reinforce bad behaviour out and replace it with the newly slightly more aligned model.

In the rest of the world we delegate whatever idea of moral agency we have about the model to a person, which may include a corporation as far as legal philosophy is concerned and depending on the type of harm created.

> The concept you’re likely actually interested in is whether or not the human brain can fully be emulated in a computer. The answer is that with enough computing power, yes.

That's kind of begging the question, because we don't actually know that, and I'm personally not sufficiently convinced of it.

Considering it's the only thing I'm directly aware of I think I can safely reject that it's a made up concept.

Ok, then lay out what it is in words that adequately covers what consciousness is across the spectrum of conscious behaviors. Also while doing so, prove that your consciousness is the same as mine. It's really easy to say, it's fuck all hard to actually prove.

Qualia or subjective experience. All the sensations of your experience, whether in perception, dreams, imagination, etc.

There's many things we can't prove with certainty, like the existence of an external world. A solipsist can doubt that anything exists outside their own mind because nobody can prove to them otherwise. But were does requiring certainty get us? Science doesn't require certainty, because empirical investigation is never certain.

Consciousness is a bucket of things. Putting subjective experience to the side, the main components could be:

- An internal model that is actively revised (the Markov blanket/s) - A continuing active resolution of inputs (either sensory or the ability to simulate "dream" - basically aliveness) - A persistent state of completed resolutions that are recursively reentered and affect the future resolutions.

Everything we might say have these abilities are organic, but a digital system could possibly be defined as being conscious if it met all those.

LLMs have an internal state, they respond to external input, they have a form of memory.

What is missing is agency; a directionality of resolution, and the subjective experience (what the conscious process "feels' like from inside.

I would suggest that those developed purely via evolution as the drive to procreate and stay alive against entropic decay.

When you stop typing, the LLM is not there wondering when or what you will type next. It responds if you ask it something, it doesn't wait around wanting to continue the conversation. There's also no loss if you stop or delete it completely.

The next phase could be adding something equivalent to dreaming (except a LLM doesn't sleep), where the LLM continues "thinking" about the conversation, and generates something additional to say without explicit prompting.

And maybe a digital system needs to know/sense/feel its potential mortality, and cares about that?

You, I and everyone else simply don't know.

I'm personally open to the fact that a digital consciousness could arise sometime in the future. Maybe it isn't anything special at all. Insects seem to be conscious too and we don't hestitate for one moment to kill them. Idem livestock.

The antenna theory is intriguing but always bugged me. Where do we get that idea that consciousness lives outside the body/brain? Where did it come from?

It's just Consciousness of the Gaps. Quantum mechanics are mysterious and are a frontier of science, so surely consciousness must dwell there.

The answer to what consciousness is does not come from anything yet known so it is natural to hope it exists somewhere just beyond what we know well. There is nothing wrong with this.

> There is nothing wrong with this.

I would say it is, you shouldn't make unsubstantiated claims based on gaps in our shared knowledge, that's very unscientific. Just because we don't understand something very well, doesn't mean we can just attribute other unexplained phenomena to it.

As for quantum effects in the brain being responsible for consciousness, there is very good reason to think it is false, no matter how much Penrose insists.

I’m not sure why you willfully misinterpret my post. What’s unscientific is to positively assert unsubstantiated claims, not to posit a theory. You also inserted Penrose out of thin air so I’m not sure you’re even engaging with anything I’ve written.

That's just another name for dualism, wikipedia has a history section about it, if you're interested:

https://en.wikipedia.org/wiki/Mind%E2%80%93body_dualism

I suspect that the nervous system acts as a distributed antenna for "picking up" other antennas.

I absolutely positively believe that consciousness is tuning into something else. Some specific arrangement of information and neurons allows it to be received.

The idea that consciousness is emergent, substrate independent, and depends only only a very complex arrangement or perhaps some kind of self referential system etc etc makes no sense at all. Then everything about the system is entirely equivalent to its external interpretations, but arbitrary external interpretations can be put onto anything. I can just throw a bag of bits on the ground and read a conscious mind out of it if I stare hard enough.

Why does this bother you? Is it an internal sense of ego that seems to be ingrained in our complex set of bits?

Really this is why it took humanity 100,000 years to reach the space age. We cannot look at things emotionally and irrationally and expect to receive rational answers. Consciousness emerges and we can see a gradient of it across different bags of meat. There are also some studies that show signs that plants can show conscious signalling behaviors, at a much reduced set of functionalities than animals.

Maybe this is just the wake up call humanity needs to know that consciousness is not that unique to us?

Also https://en.wikipedia.org/wiki/Sorites_paradox , when does a pile become a heap?

I doubt it bothers me in the sense you have in mind. What bothers me is the desire to actually understand it and always see all the smart people stuck on a non-starter. And it’s the first idea most smart people have about what consciousness must be. I also thought this when I was a teenager and after reflecting on it for many years decided it does not work.

I believe animals have consciousness and probably plants may have some kind of qualia so also may have it, so your charges against me seem misdirected. If an AI system happened to have qualia I believe it would depend on its substrate (which might be something we don’t understand yet), not because of how intelligent (how complicated it’s processing). And I believe a different substrate running the same processing may not have qualia. I don’t think we randomly chose the right substrate in the 60s-70s.

And I don’t argue against the idea that concepts like heaps are emergent. You’re making a logical error in assuming you know what consciousness is and thus I don’t know what emergence means. No, I do not think consciousness is emergent in the sense of a sand pile.

And before the army of false dichotomists show up, my position is not biological chauvinism or whatever your LLM has told you about.

Now you've put yourself into a trap. "Consciousness is emergent" but also "Consciousness is not emergent". Only one of these statements can be true. Life is that sand pile. We were a pile of chemicals that got hit by lightning or whatever billions of years ago, and from those uncountable chemical and electrical reactions nature evolved a programming machine that reliably produces conscious entities from non-conscious materials. Again, 4 billion year old sand pile.

Personally I reject the hard problem of consciousness. People keep having to try to invent magic, god, or unfound sciences for an issue that is simply complex but right in front of us.

I didn’t say consciousness is emergent? Given some fundamental mechanism and substrate which is the essence of consciousness sure there will be emergence on top, but I’m not sure how you’re getting me saying consciousness is emergent when I clearly expressed the opposite.

You can reject the hard problem, but that just amounts to giving up with pride.

Reply on news.ycombinator.com