Comment by orbital-decay | Hacker News Reader

Comment by orbital-decay | Hacker News Reader

>The idea that the same consciousness algorithm can be run on a variety of different substrates makes no sense when the substrate in question—a brain—is continually being physically reconfigured by whatever information (or “algorithm of consciousness”) is run on it. Brains are simply not interchangeable, neither with computers nor with other brains.

This is kind of self-contradictory. Then humans aren't conscious? Or each has their own consciousness? Then why not the machine? Not sure what's the point being made here. Yes, the states of a human brain and a transformer are absolutely incompatible (humans at least share the common architecture), that's why any attempts to map model's "emotions" to humans' and the entire model welfare concept are pretty dubious. That doesn't prove there's no (or can never be) consciousness in that, though.

That's the most coherent argument from the entire article. It criticizes the Butlin report in particular and extrapolates that to "never", while ignoring modern takes on that (e.g. interpretability studies showing vague similarity of both on a level deeper than just the language) and any possible future evidence.

In a sense the title is right, nobody ever formally defined consciousness, so you and I and anyone else are free to make almost any argument and spin any narrative according to our beliefs and it will be true! Ill-defined terms and baseless solipsism are the main problems with all these discussions. Good thing that in practice they matter as much as the question whether a submarine swims.

The substrate argument is the wrong hill for Pollan to die on. The stronger version isn't "meat vs. silicon" — it's that brains are value-making machines operating under evolutionary pressure, and no current AI architecture has anything analogous to that. You can simulate the outputs of valuation without having the mechanism. The question isn't whether consciousness can exist in another substrate, it's whether you can get there without the thing that actually drives human cognition: spontaneous assignment of moral and survival value with no prior programming.

AI is an extension and acceleration of so called "evolutionary pressure". But so far AI models lack both agency and consciousness, and do not "experience" this pressure, though they are entirely defined by it. They can also explain this relationship to you.

That's true in the same sense that agriculture and nuclear weapons are extensions of evolutionary pressure. Everything humans produce is, by definition. But it empties the term of any useful meaning.

The distinction that matters: evolutionary pressure operates through differential survival across generations, where the organism has skin in the game. AI models are optimized via gradient descent on loss functions that humans define. That's artificial selection toward human objectives, not evolutionary pressure in any meaningful sense. The model has no stake in the outcome. Nothing is at risk for it.

You actually make this point yourself in your second sentence: they "lack both agency and consciousness, and do not experience this pressure." I agree completely. But that's precisely why the first sentence doesn't do any work. If they don't experience it, then calling it evolutionary pressure is metaphorical at best. And the metaphor obscures the exact gap we should be paying attention to: the absence of anything at stake.

It's a fine argument, just rough around the edges:

Human brains use redundancy and the physical independence of neurons to build new pathways over time.

Current LLMs have no redundancy and brittle weights. Their technology and architecture fundamentally prevents them from learning.

I think our understanding of consciousness is developing as we build new edge cases. We have a machine that understands and reacts, but can't learn, grow, or "be" over time in a meaningful way.

But how is neuroplasticity relevant to consciousness, whatever it is?

>Their technology and architecture fundamentally prevents them from learning.

No? There's in-context learning which is actual learning, it's sample-efficient, and the result can be stored for a learning pipeline. Yes it's ludicrously crude and underpowered compared to neuroplasticity, but that's another question, there's nothing fundamental about this.

Reply on news.ycombinator.com