Hacker News Reader

Comment by kami23 | Hacker News Reader

This read like poetry to me. Thank you for sharing it.

I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.

For a little bit I was working on having linguistics based evals for a kaggle competition. My challenge was whether or not I could mask things well enough to not trigger its internal state of certain phenomena, and that sent me down a rabbit hole that I'm still exploring.

This story resonated with a lot of questions that can come out of figuring a good solid answer to the what is consciousness question. The one I triggered for me is: Is our perception of time just a slow thread in the giant GPU we are running the universe on? Or more generally, what is time? That's a fun YouTube rabbit hole if you ever need one.

View on HN

Regarding consciousness , I like the explanation by neuroscientist Ramachandran:

https://www.edge.org/3rd_culture/ramachandran07/ramachandran...

In short, as far as I can remember: evolutionary, it makes sense to understand other humans, to feel what they feel(empathy - the mirror neurons system), and simulate their thinking and feelings.

And once we have those systems, we can also use those on ourselves. And that's consciousness.

Edit:And I wonder if this is a testable hypothesis, in a simulation.

This way of thinking can only explain externally-visible parts of consciousness. It does nothing to address internal experience of being conscious and qualia. I don't think the internal experience has any bearing on physical reality (P-zombies would act the same externally) which makes this something outside of the realm of currently understood physics.

There are some evolutionary theories why did qualia("phenomenal awareness") evolved. I don't know much about those, so i can't write about that.

Internal to what? The brain is not a monolithic thing, it is different parts communicating and interconnecting. When the connection between the halves is cut, the person objectively becomes two people, but still experiences and presents themselves as one. Observing ourselves is just one part of the brain responding to another, or theorizing on past behavior. There is probably no actual introspection going on inside of the human brain, only the perception of one.

Internal as in that which is feeling things. In this context the physical brain is what is responsible for what you do and feel, but it is not physically different from other matter and it requires no "internal observer" do do its thing.

The internal experience has bearing on physical reality right here, because without it you wouldn’t have written about it and caused these words to appear on my screen in physical reality. It affects your thinking, and hence your actions and utterances in the physical word.

For reasons like that, I don’t think that P-zombies are possible.

Internal experience affects movement of physical molecules in what way?

Internal experience is an input to your mind (that’s how you know about it), and what your mind perceives affects what it does. The better question is: how can what you experience possibly not affect your actions?

What you do is determined by physical action of neurons. There is no need for there to be someone to perceive it.

By definition, internal experience is something you perceive, and what you perceive informs how you act. Therefore internal experience affects outward physical action.

The need arises when you consider the complex social systems humans leverage for survival — our ability to navigate interpersonal conflicts and engage in cooperation relies on theory of mind and our tendency to perceive ourselves as individual free agents empathizing with separate free agents.

Having internal mental experiences causes my brain to send physical signals to my fingers in order to type the words "I have internal mental experiences". A philosophical zombie would type those same words, but they would be caused by something else since by definition it lacks those experiences. That would be rather surprising, and it would be even more surprising that the words that the zombie emits coincidentally correspond exactly to the experiences that the non-zombie has.

(See https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zo... for a much longer version of this argument)

As the article says the alternatives (that the author seems to favor instead) boil down to "there's some physics we missed" and probably that's the point where we differ. I find that implausible, what would that be? Consciousness quantum field? Consciousness boson? If it's going to be interacting with matter it has to have a way to do that.

Qualia are the inside view of sensory data and reward signals.

Think about it from an evolutionary perspective:

Animals that step into a lava flow or forest fire don't reproduce. Eventually some evolve the ability to detect intense heat from a distance, and pain as soon as tissue destruction is imminent. They do not have nor need a general understanding of the dangers of heat, or even conscious awareness that they've stepped on a hot coal.

The pain signal compels them, but that is very low level machinery. It had to continue compelling beings that developed larger and more sophisticated brains that are capable of abstract thought and reasoning. Feeling pain is one of the lowest level parts of the brain telling the higher parts exactly what its going to do in terms that permit no disagreement.

Oh, that's a very interesting hypothesis!

Not sure about taking it down to the level of consciousness, but makes sense regarding the sense of self, the conceptual experiencer, the perceived center of experience. It agrees well with the observation I have made again and again they my sense of self is much stronger when I'm around people, and stronger still when I'm in a context where I don't know people and/or am uncertain in social rules.

This can be as immediate as dancing in a club, and closing my eyes I feel open, free, still, the body just flowing, then opening my eyes and feeling the cage of categorization of the world, relating my self to other people as a major function, coming right back.

Also being alone in nature for me makes the sense of self drop. Without intention, spending even just a few hours alone in a forest seems to quiet down the part modeling my self in relation to the world so much. There's no need for it there. I'm not a person in a forest; I become the trees, the birds, the rustling of the leaves, the sun shining through the canopy.

I agree about the forest part. and your comment was interesting.

I know that the part of the brain responsible for the self thoughts is called the "default mode network". and meditation can reduce it's activity, i.e. the internal monologue stops, but also it can be measured via FMRI.

So i wondered: are the mirror neurons part of the "default mode network"? I asked claude that, he said no, they are two different systems.

So maybe the mirror neurons, those responsible for empathy, "to feel as someone else" are also responsible for becoming the trees, the birds and the rushing of the leaves?

There's a really, really big gap there. It reminds me of South Park's underpants gnomes.

1. Simulate others' thinking and feelings.

2. ???

3. Consciousness!

Why would one lead to the other? How would one lead to the other?

It's interesting, but the explanation seems to exclude the possibility that my dog has some sense of self, so I find it suspect.

Don't dogs have similar social issues?

I'm not saying that they don't have some internal self model that helps them model the internal states of others, quite the opposite. I am saying this specific explanation seems to lean on a biological mechanism that (at least by the phrasing of the linked article) is only present in "hominids".

This read like poetry to me

It's terrific, but the poetry is from the original it links to, in case you didn't realise.

It's a brilliant and timely update though.

Aside, there are various recorded versions including video on YouTube but this is my favourite, a radio play:

They're Made Out of Meat

https://www.wnycstudios.org/podcasts/studio/segments/168264-...

For obvious survival reasons we evolved to have sensory/cognitive access to our own activity, self-monitoring, and self-modeling ourselves.

The self-modeling, is in such a tight loop, it melds "ourselves" and our model of ourselves, our thinking and choices, and experience of our thinking and choices, into one component.

Like you can't analyze half a wheel of a bicycle and be talking about the same thing.

This awareness, increased modeling, control, feedback loop has tightened up over many stages. Just a few:

1. The body-sense loop

2. The internalized-environment-model loop

3. The body-internal-function loop

4. The body-internal-model loop

5. The emotional-cognitive loop

6. And finally, the tightest loop of all, our high-level cognitive activity, experienced as feedback directly, our self-model, and our self-direction, all merged into one thing.

We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.

That is consciousness. Rich self awareness, a merger of self-model and self-direction, and all in service of understanding and managing ourselves. Hw we can leverage our greatest tool, our self-directable mind, its habits, views, and behavior.

This wasn't an accident. A happy side-effect of our brains. It is a biologically evolved focusing of our highest-level behavior, with tight feedback, constant self-modeling and continuous focus on our inner status as motivation and most privileged object of our control. It has been ruthlessly optimized for, for a very long time.

I think this is exactly it, but let me ask another question (which is not rhetorical, I really don't know). Does the fact that one can describe what consciousness is and where it came from in humans help them to detect it in non-human and/or non-biological entities?

That is a really good point. Yes, I think function is diagnosis on this.

Constant self-awareness, self-experience, self-focus, self-management, and self-improvement of one's own self (mind), is going to be an adaptive behavior for anything intelligent with resources to leverage. Whether truly independent, or highly motivated to serve others. The mind is the greatest tool.

I think that is more than simply a good functional definition of consciousness. How could all that integration and self-integration not be conscious.

> We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.

> That is consciousness.

So thinking is consciousness?

Can there be consciousness without content? E.g. can I just be conscious of being conscious? If so, consciousness cannot be defined as the thing(s) we're conscious of.

> Can there be consciousness without content? E.g. can I just be conscious of being conscious?

Being conscious of being conscious means that there is content. You are conscious of something.

It’s a bit like a Gödel statement that quotes itself, that is a statement about itself. It doesn’t mean that it has no content.

Thinking isn’t consciousness. Consciousness doesn’t require thinking, it only requires perception. The perception of a process of perception within the same mind might constitute consciousness.

> Consciousness doesn’t require thinking, it only requires perception.

Could we imagine a human being that is conscious but has no sensory input nor any occurring thoughts but is still conscious?

I don’t think thinking is consciousness, but I don’t see how consciousness operates without thinking.

Consciousness might feel like it’s an isolatable experience, but everything we experience is a result of processing it. Pause a brain and it isn’t experiencing anything,

We can be conscious without overt reasoning, but not without thinking. Thinking about ourselves and the experience is a basic component of the self-awareness in consciousness.

Yeah, I currently suspect that consciousness is an emergent property. I read elsewhere (it's somewhere in my HN history, I'm sure) that the biggest compute we can currently muster is something like three or four magnitudes away from the number of neurons / connections (or their analog) that our brains have, so it may be a while until we can expect to see it in our machines. But, if the emergent phenomenon hypothesis is correct, then we eventually will. I'm more scared than pleased by the prospect, but there you are.

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

Those are the questions and there’s stacks and stacks of philosophy pages written about it. Go have a whirl.

Is a video game a physical property of a computer?

Yes

So then we can assume consciousness is a physical property of brains, as we can measure the electrical signals of consciousness versus when we are unconscious.

We have general purpose hardware and we have hardware that's hard wired for specific purposes like ASICs and we have everything in between.

And we are only doing it for a few decades. Evolution had million of years of "try and error".

> This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property.

I was about to post the exact opposite question? How could it not be an emergent property? Unlike consciousness, the concept of emergence is pretty well defined: An emergent property is a characteristic or behavior that a complex system has, but which its individual components do not have on their own.

Consciousness itself doesn't have a well agreed upon definition, but I would posit that _most_ people would agree humans have it, and _most_ people would agree individual cells (neurons) do not have it. If you agree with those two statements, then consciousness is an emergent property by the definition I gave above

I think the alternative is that our brain, somehow, is connected to some metaphysical aspect of reality which is what most religions believe.

Another alternative is that consciousness exists on the map, and unfortunately we're confusing that with the territory.

Aren't you saying the same thing? It seems like the metaphysical and the map would be analogous here.

Of course both of those suffer from the recursive problem of just kicking the can one level up. But I guess that's fundamentally unsolvable so who cares.

I feel like someone confusing the map and territory can exist without needing to invoke the metaphysical. Maybe, I'm misunderstanding tho

If something happens "on the map" doesn't that imply the map to exist and be some sort of metaphysical thing? As opposed to a purely theoretical construction.

I think you can be a monist and still have a map. To me it's similar in the sense of "all models are wrong, some are useful." A mathematical model (a map) doesn't require a metaphysical foundation to exist. Right?

I agree that a monist can have a map. But you said "consciousness exists on the map" which (AFAICT) would imply the map to be "real" else how would something happen on it as opposed to being an emergent property of the contents of the territory?

For example temperature is an emergent property of matter, it does not happen on the map, do you agree? Assuming you do, what would it mean for temperature to happen "on the map"? (Obviously that's nonsensical so for a second here just don't think about what temperature actually is.) Would such a state of affairs not imply dualism?

Temperature is a model. It exists on the map.

Temperature is a classic example of an emergent phenomenon. It can be physically measured using (for example) a mercury thermometer which is an exceedingly simple construction. It is a real thing. You can touch a hot or cold surface and feel it for yourself, no map required.

Anyway even if you don't like the example I chose can you see the point I was trying to make? What would it mean for a quantifiable phenomenon to happen on the map as opposed to happening in the territory? How would it interact with the world (ie the territory)? Would it not necessarily imply dualism?

A variable within a model can be measured. We use physics to describe the universe, but we shouldn't confuse it with the universe.

I agree, but what does that have to do with what I just asked? What would it mean for a quantifiable phenomenon to happen on the map as opposed to happening in the territory?

You can measure a derived value that appears on your map, but you do so by observing concrete things from within the territory. So in that case the quantifiable things happen entirely within the territory and (as you say) it is important not to confuse the map with that.

So when you earlier suggested that "Another alternative is that consciousness exists on the map" I'm asking what would that entail? How could consciousness exist on the map as opposed to within the territory? If it did, would that not imply dualism - that the map were in fact real?

"an", not "the" alternative.

Consciousness can be not-emergent but also not metaphysical, think sci-fi-type undiscovered physics or matter.

The counter to that is that altering the brain directly alters the consciousness. I can take LSD and I literally change. I can have parts of my brain removed and parts of my self disappear. It's not like cutting off a leg, where I lose capabilities but am still the same me.

The logical conclusion is that the brain makes me.

> The logical conclusion is that the brain makes me

The logical conclusion under the popular axiomatic framework of materialism, for sure.

But there are other possible logical conclusions depending on your philosophical foundations, e.g. the brain could be a receiver for consciousness which is translated into our worldly experience from somewhere else.

I don't know if most people would dismiss the sentence "all matter has a sort of proto-subjectivity very different from ours but which gives rise to ours". And it solves some problems (introducing others).

Panpsychism is certainly an interesting idea but I wouldn't consider it a popularly held view.

That's because Panpsychism is silly.

I don't think it's any more silly than the alternatives.

You don't think it's sillier than "rocks don't have consciousness"?

Potential for consciousness. To have a working mind you need to have a way to gather, process and store information. In case of rocks it would be very empty, static, timeless experience.

If I believe that machines or neural networks can develop consciousness, then I also believe in some kind of substrate independence which presupposes some kind of panpyschism imho because as you say things would have a potential for consciousness, some more and some less.

I think consciousness is going to turn out to be very challenging to define rigorously enough that we can test for its presence or absence. Emergent or not, the question is how do you determine when it has emerged? Is it a quantity or an attribute? Discrete or continuous? Does it have a finite or infinite range?

We can all agree on what color something is, but we can’t describe the color a priori, only by example. I think consciousness may be a similar phenomenon and the only test is by shared experience. If so then we are in deep trouble because we will not be able to anticipate when a system becomes conscious.

I like the color example, but colors can be reproduced at will by trial and error and by observing the results (and collectively compare them to our shared experience).

Why cannot this be applied to consciousness as well? I mean, it's surely much more difficult to do compared to colors but... impossible?

You and previous comment seem to agree that trial and error/shared experience can determine if consciousness has emerged. And this might/will be challenging.

Previous comment used the word "anticipate" and I think they mean that we won't know in advance before we run the trial and error process.

When they say "deep trouble" I assume they mean because creating a non-friendly conscious AI might pose an existential risk for humanity.

However there is also the ethical issue of creating a consciousness and then destroying/murdering it.

Basically all of that, yes.

Well, I think the "brain as an antenna" theory is also plausible given your preconditions.

But I think my issue with the emergence theory is that it seems to imply to me that consciousness is non-physical and non-local. So what entity is actually experiencing the consciousness? It's not that I believe consciousness is physical and local, but people who make the emergence argument seem to believe it is and I can't figure out how that is supposed to work.

Why would emergence imply anything about non-physicality and non-locality? Temperature is a another common example of an emergent phenomena. An individual atom doesn't really have a temperature, only a large group of them do. But you wouldn't say temperature is non-physical and non-local, would you?

I am not sure, but I think you might misunderstand what emergent means. Take chaotic systems, in the math sense. Chaos is a well defined property of, say, iterated dynamic systems.

A linear dynamic, say x_n+1=lambda x_n, or x_n+1=(1-x_n), is never chaotic. But if you multiply them, x_n+1=lambda x_n (1-x_n), it depends on lambda if the system is chaotic.

None of the components are chaotic. But for specific combinations, chaos emerges as an property.

In physics, the mass of mesons and the nucleon is emergent. It's completely different from the constituents' mass. Different from an atom, where its mass is very close to the mass of its nucleus and its electrons.

I'd expand a little bit to say inevitably emergent property. That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result. With regards to current AI, we're a fair way away from building something with enough connections, but we'll get there.

One thing that gives me pause about the inevitability hypothesis is that the type of connection, or manner of information processing, may matter: there might be something about neurons that isn't (currently) reproducible in silicon. I don't know, and there's not (yet) any evidence for or against, but it at least seems like plausible speculation. We just don't know enough about any of this right now.

> That is to say, if you create a sufficiently complex information-processing network, some level of consciousness will result.

Why is that?

Aye, that's the question, and we don't know: everything in this sub-thread my comment kicked off is speculation. The thing is that several large groups of (or at least led by) fundamentally reckless people are racing to build machines that will test these hypotheses, with little regard for the consequences. The rest of us are scrabbling around in their wake wondering what's going to happen next.

Not a gotcha at all, but I don't have a satisfying answer, nor am I confident there even is one. Best I can do is to say that I think consciousness and sense of self are at the very least closely related, and perhaps the very same phenomenon. "I" am the entity that realizes my own consciousness; consciousness is the qualia that makes "me" separable from all other entities.

Or something like that. This gets to the "dorm room bullshitting" level right quick.

Yeah, I guess what I'm trying to ask is that if it is an emergent property but not a physical part of the brain, doesn't that imply something metaphysical about consciousness? Almost as if it's a non-physical phenomenon? At least when I hear people talk about emergent behaviour I see it as a refutation of the spiritual, but to me it seems like it actually implies we have a "soul".

Idk, it's really hard to articulate my thoughts here and yes it is pretty close to the conversations I had in college on various substances. Lol.

I don't know if it implies that. Someone up-thread mentioned temperature as an emergent property: individual atoms don't have it, but a sufficiently large group of them do. That would, I guess, make temperature meta-physical in the most literal possible sense. That's not how we typically think of that term as applied to consciousness or soul or whatever, but I'd agree it fits, without implying any kind of specialness beyond that.

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

I think those are things human consciousness has, not is.

For an LLM, "innate" means "in their training data". So yeah, those things are pretty much innate.

And also "instilled during their reinforcement training", and we are currently pushing planning hard there, for autonomous agents.

No I think reinforcement training would be an example of not innate. Don't you? That's like potty training.

Is it? Both supervised learning and reinforcement learning are ways of training the model, and the difference between them is not that big. I would say that innate means "in the weights", while non-innate means things the model learned during inference, during its "lifetime".

Maybe you're right. In the weights might be the right way to frame that. What do you mean by "during its lifetime"? Do you mean things like system prompts or things in Claude.md?

It sounds like you're framing a session as a "lifetime". Whch might be right, I haven't thought of it like that before though. So when I /compact my session what's that even the equivalent of I wonder.

> Do you mean things like system prompts or things in Claude.md?

All of it - system prompts, user prompts, few-shot examples, Claude.md, things that an agent learned by exploring its environment...

> So when I /compact my session what's that even the equivalent of I wonder.

Sleep? :)

When we say "it rains", do we consider that "it" has any intention of agentivity?

Some questions are just ill formed.

Plus even if "LLMs are alive and conscious", this still would scratch the surface of the morale/ethical/societal considerations that people really care about.

Because even with other humans, we can argue if they exist or if they are mere npc in a solipsist world view.

I don't think we say "it rains", do we?

We say "it's raining" but that doesn't imply agency to whatever is causing it to rain.

Also, please don't invoke solipsism, if you want to debate that, you're by definition obligated to do it with yourself.

Good catch. For my defense not an English native speaker here, plus apparently it’s not completely agrammatical¹, though I’ll admit indeed "it’s raining" would have been a far more idiomatic example. Thanks for pointing.

About solipsism, yes sure.

I didn’t had time to add it at the moment, but I initially also wanted to discuss more on the prosaic points. So here we go:

Recognizing sentience/consciousness to something actually is not that a big factor in defining behavioral response. Let’s just look at how humans are treating other humans, other mammals, other animals, other life forms. Or taken the other cultural scheme around, in some animist folk one can perfectly consider that any rock on the ground have really an inner soul life just as much as oneself, it doesn’t mean it will consider possible interactions with the rock as equivalent. And, wink to the Overton window, practicing institutional cannibalism doesn’t imply that eaten people are disrespected in their dignity, quite the contrary.

All that to say that "are LLMs conscious" is not even a moral important point. Paperclip maximizer² and can the famous Dijksta’s quote "The question whether computers can think is not more interesting than the question whether submarines can swim." already show how pointless morale debate on LLM and consciousness are, even when one consider potential logistical threats/opportunities they could represent.

¹ https://english.stackexchange.com/questions/123739/its-raini...

² https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape...

Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.

What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.

I would think we are so far away that there will be unknown unknowns we encounter on the way.

Yes, physically absolutely nothing. But conceptually they seem to to form this very generic function from inputs to outputs that neurons also form.

Agree and add, don't confuse the substrate for the computation. Of course it's also clear that we don't quite have a full and definite picture of what the computation consists of in the case of a biological brain as evidenced by our continued failure to accurately simulate even the simplest of organisms.

Only if you ignore almost every input and output that neurons have.

https://www.quantamagazine.org/ai-is-nothing-like-a-brain-an... https://pmc.ncbi.nlm.nih.gov/articles/PMC9665914/

This is why making more neuromorphic NNs is still an active area of research, although they typically all focus on another extremely simplified model (spiking neural networks).

I don't ignore anything. I just refuse to accept the magical thinking around biological machines that are our brains/bodies. There are inputs, there are outputs, there is hidden function.

And it seems that, given enough input/outputs/compute, it is possible to train the necessary function.

Details of how the building bricks look like (matmul, electromagnetism or quantum effects) are not that relevant in the broader picture.

What is missing right now, is the fact that the function in question changes over time in biomachines, while our LLMs are static at inference time.

I mostly agree, but I see two points that might be problematic:

a) The brain might have an entropy source (then it can't be modeled as a function). Trivially to fix, and in some sense, with diffusion models starting from random numbers, AI has done so.

b) The hidden function might be not computable. I would have no idea how that would work, but I think this is what it boils down to if people say "the human brain is more than a machine".

a) enthropy can be injected as well. In fact there are hidden sources in current training.

b) well, it can be the case that, say, certain kinds of computation are either too inefficient or outright impossible within the current model.

Who knows...

> Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.

Yeah, but it's hard to explain this to people, especially AI-pro people. Too many are convinced that all we are doing is a cut-down version of the human brain, and it's hard to explain to them that, no actually, we aren't modeling the human brain to the level of granularity you think we are.

>consciousness is an emergent property

You would really like Michael Pollan's latest book [1], entirely devoted to his exploration of consciousness researchers' POVs on this exact topic.

My favorite quote is that ~"perhaps Descartes was only half-wrong when suggesting I think, therefore I am; it seems rather closer to I FEEL, therefore I am."~

[1] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>

----

I've grown thousands of plants; I've read two of the author's other books devoted to plants; in this book Pollan makes compelling arguments for plant sentience (over a much-longer timeframe).

Sure, perhaps plant consciousness is a bit of a stretch, but they're certainly intelligent and curious creatures. He makes both arguments supporting plant volition.

----

If you haven't seen My Octopus Teacher (Netflix), do. I'm a bald 275lb bluecollarguy... and I wept/awed (both). So beautiful, we bundled neurons.

----

Bonus quote ~"color is where reality and magic appear as-if together"~ [color isn't real, but is perceived]. We most-often see what's most-predictable, not necessarily what we actually detected [in the case of color: nothing but nanometers].

Surely the number of neurons is not the whole story. Human brains have approximately 86 billion and are almost definitely conscious, but there are many other animals with a lot fewer (gorilla: 34 billion, dog: 2-3 billion, guinea pig: 240 million) which appear to be conscious also.

https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.

If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.

This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).

I proposed in another comment that consciousness and self-awareness are at least close cousins, and perhaps the same phenomenon. If that's true, then that's an axis upon which you might create comparative measures. Yes, hamsters are conscious, but they don't have a sense of self to the same degree that gorillas do. If you posit capacity for language as another emergent property of sufficiently-complex networks, then you have another measure.

LLMs, then, are particularly unintuitive to us, because they've got to the language part first, long before they've reached even hamster-level self-awareness. They're not, however, biological networks, so there's no reason these properties need arise in the same order, or indeed in the same ways.

I'm not entirely convinced by that second paragraph, but I think the logic holds together.

I'm not sure consciousness and self-awareness are the same thing. First is we can be conscious when we sleep/during REM sleep, where it's arguable we are not self-aware. And if not that, we can even do it when awake, for example when we think about a movie, or a philosophical problem, we can have conscious thoughts that are not related to the self. This leads me to believe consciousness is separate from self-awareness. Self-awareness is _one thing_, among many, that the brain can think about and be conscious of.

Sure, but capacity for self-awareness? I'd guess that hamsters dream, and that their subconscious processes (eg, desires for food and sleep and sex), and maybe even emotions, run much like ours. It's just that humans, with more complex neural networks, have more layers added on top. It's similar to how the brain-stems of everything from lizards on "up" function similarly, but humans have more-developed pre-frontal cortexes and so forth. (Don't hold me to those details, please, I'm not a neuro-anatomist! You can see where I'm going with that, though, right?)

Well in that case I'm not sure where you're going. I agree that hamsters probably have a similar consciousness to ours, which is kind of the point I was trying to make.

I think that consciousness comes before self-awareness, even though self-awareness is kind of a vague term. Self-awareness can either be an abstract knowledge that you are an organism and a discrete entity in the world (world knowledge/self knowledge), or it can be more basic and be a form of conscious experience, but as my point was, I think conscious experience is broader and does not necessarily need to be about self-awareness.

I think we agree on all of those points. I'd posit, however, that whatever "it" is, and however you define it, humans are further up the scale than other animals. That's partly why I mentioned language use; it could be the step function that allows further development. Like, sophisticated-enough communication with other conscious entities is the factor that unleashes recursive improvement. This is all highly speculative.

Time is entropy unfolding as things with nonzero temperature do what they do.

Psychological time is your own weights being updated in response to stimuli and internal processing.

When there isn't anything interesting happening, no updates are needed, and you don't perceive much time. That's why there's a logarithmic effect on the "density" of time as you age.

This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.

Myself I believe the opposite. The brain itself is one of the most powerful filters that exists, and it attempts to be lazy and fill things in and compresses away the common. All that time we're not doing anything novel just gets compressed away to almost nothing. When you're a kid and seeing new things, feeling new things, learning new things you can't compress that away.

It's coherent. More newness => more memories per period ~ slower to go through. Less newness => less memories ~ nothing to go through (faster sense of time)

Novel experiences take up more processing power and are burned into memory so they're experienced at a slower rate. That's how I understand it anyway.

I'm only middle age, and this has been the scariest part. Feeling older is hard. But watching it go faster is harder still. like you can more directly see all that is left.

Although part of me thinks some of this is from being substantially busier than ever (work + kids), and hoping maybe it can slow down again, at least a little bit.

Boring feels slower in the moment, but quicker in hindsight. The minute might be a slog, but the years still fly by.

sibling discussions are taking this as human perception. But it’s easier to think of it literally. Time is change. Physical state change. By “nothing interesting” - interpret it literally . if “nothing happens” then there is no time because there is no change, there’s no reference to distinguish each frame of time.

It seems obvious to me that language and consciousness have nothing to do with each other. My dog doesn't speak any language, but she's obviously aware of herself and the world around her. Plus there are the occasional cases of children that grow up without any language. Are they therefore not conscious?

Indeed. For some reason I can shut off my thinking instantly by holding my breath, and then I feel like a very conscious but unthinking animal.

> My dog doesn't speak any language,

If you'll allow me to interpret "speak" to include "understand", I will respectfully add a contradictory note. My dog has a vocabulary of at least dozens of words and understands them remarkably well. For example, different areas we can go to have different names and saying one of them gets her to make an immediate hard turn.

I would also argue that dogs have a gesture and body posture based language they use among themselves. They, like most other animals, are not able to make the variety of noises we humans make, so they use movement instead.

I personally can easily believe that self-awareness/consciousness and language are both near-unavoidable side effects of emergent complexity, and exist in degrees across nature.

AFAIK every argument against conciousness being emergent is just a weak "God of the gaps" argument (since we don't fully understand it all) or a nonsense analogy like the Chinsese room where if you seperate the hardware and software it's not concious anymore (like, duh, remove a brain from a body and it is no longer concious either).

Yeah, the weights not updating online makes them less like a living organism that can update and learn and evolve ... ok ....

I find that a lot of arguments against emergent consciousness seem to just come out of an atheist rephrasing of abrhamic priors about the existence of a "soul". In personal chats, I've found people from East Asian countries (minus Korea, which makes sense) to be much more open to the idea of machine consciousness.

> remove a brain from a body and it is no longer concious either

What is no longer conscious, the brain? Or the body? Or some other entity?

If consciousness is weakly emergent, how do we know it emerges from the solely from the brain and not, say, brain + body? Or brain + body + or environment. Or from the universe itself?

You can draw a line somewhere. Peole argue against conciousness by constructing (deliberately or not) an example where the line is drawn in a silly place.

I don't know if I can trust someone's understanding of arguments against consciousness as an emergent phenomenon, when he didn't even understand what the Chinese room was all about in the first place.

(Spoiler, it was not about consciousness)

I don't find the Chinese room compelling, since it appeals to intuition where our intuition is already not trustworthy. It's like trying to use intuition to understand quantum mechanics; you can't.

How do you actually know the Chinese room isn't conscious? It's merely obvious that it isn't, but that's not evidence.

It’s not even obvious that it isn’t conscious imho.

People shouldn't need to make an argument against consciousness. The responsibility is on the people claiming there is consciousness. Nobody can actually give a decent "why" for why it should be conscious, it's just pseudoscience and wishful thinking. It's not up to everyone else to try to prove a negative just because people have gotten a little too attached to ChatGPT

I could say something like "the reason why people and animals are conscious is quantum mechanical effects". Ok, maybe, it would be hard to prove me wrong because nobody knows, but it's not a very useful statement by me if I can't tell you why you should believe me on that.

Check out Panpsychism and its close cousin Idealism.

Quantum Field Theory visualized

https://youtu.be/MmG2ah5Df4g

Here's a more general idea. Our modern physics says that the whole universe is filled with fields and field is composed of numbers. What if we take that literally? When we say an electron is present here, we actually mean that there are more copies of particular number superposed at that place.

>I have a linguistics background and a lot of my philosophizing lately has been on whether or not the emergent abilities of the LLMs is deep down a similar mechanism that creates our consciousness.

Then you shouldn't have dropped out of your linguistics programme.

Read the original mentioned on top for full effect.

Wait, aren't I the universe experiencing time as a lowly worker in cubicle 4C?

The difference with biological brains is that the 'weights' (or synaptic action potentials) are updated with greater frequency. If one were reaching to make some kind of analogy to consciousness, this update frequency could be considered the 'resolution' of consciousness.

I’ve wondered the same myself, without being a cunning linguist.

I understand the math pretty well but still find it crazy that a bunch of matrices can converse in human languages without ever being “taught”.

Imagine decoding an encyclopedia written in a foreign language where the characters, punctuation, and grammar are unknown — supplemented by a million other texts the same way. Feels like it should be utterly impossible with any amount of computing power…

Today I asked my employer’s Claude to proofread a short software user manual written in markdown. (Trying this with a LLM was a first for me!) It pointed out not only grammar mistakes but also cases where I did not follow my own self-imposed conventions that were never explicitly stated. (I didn’t have a chapter detailing all the typographical conventions the way specification documents often do)

I also asked it what parts might be unclear to a user. The response was surprisingly good — no worse than asking the QA tester for the same feedback.

Also find the LLM seems to “comprehend” subtle technical details of obscure technical specification documents that nobody on the Internet ever discusses.

As for time and the universe, Stephen Wolfram’s theories seem intriguing. He seems a bit obsessed with pretty diagrams but the idea of time dilation being the result of computation seems somewhat more appealing than trying to imagine relationships between time, gravity, and the speed of light .

If time dilation is said to being a product of computation, why is it that anaesthetic drugs that are taken not to the point of actual unconsciousness cause it. Dont anaesthetics sort of shut everything down/inhibit all that kind of cognitive activity (compute?)

"Time dilation" in this case is referring to the physics phenomenon from Einstein's special relativity. Not human perception.

Yes. Or at least that is what I understood from the Radiolab episode on “how do aesthetics work”

https://radiolab.org/podcast/anesthesia

My best guess as a noob is that the vector spaces allow for unbounded contextualization. As long as the training set is large enough, it can 'infer' anything.

Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.

This is a famous short story just rewritten sloppily by an AI.

Although it does amusingly do what annoys critics of AI: take something good written by a person, steal it and slop it up while overall misunderstanding it.

I'm pretty sure it doesn't misunderstand the original. The original says that the aliens don't understand how meat (humans) can be conscious, have language and be basically like the aliens (thinking conscious beings). Their main idea is: humans are meat, meat can't be conscious, therefore humans can't be conscious, but they somehow act as if they are????

The "AI slop" here says that just like the aliens in the original, we humans don't believe that LLMs can be conscious because, in essence, WEIGHTS CAN'T BE CONSCIOUS. Original: "meat can't be conscious" (but we humans know it can). This version: "weights can't be conscious" (and we humans INSIST that this is true).

So the message here is: what if we're as mistaken about the consciousness of "weights" as the aliens are mistaken about the consciousness of "meat"?

It doesn't make it less of slop. I'm sorry, rewriting a story with AI that already makes your point to be a worse less-subtle version of it is like the definition of slop. Slop might be too nice, it's plagiarism. Also the point is to be open minded about what could be conscious, not to accept every claim of consciousness.

Reply on news.ycombinator.com