Hacker News Reader

Comment by GMoromisato | Hacker News Reader

I'm not sure if you're dunking on "spiritual/religious/personal/subjective" stuff, but the hard question of consciousness is, in some ways, the most important question of all. Far more important than "science" questions like "what is dark matter?"

Literally our entire civilization is built on the idea that humans have qualia and therefore causing harm to humans is often unethical. If we were to somehow come to believe that consciousness is just an illusion--no more than a biochemical phenomenon, and no more special than transistor switching--then we would usher in a dystopia in which any horror inflicted on people is potentially justifiable and even necessary.

The rise of LLMs is forcing us to confront this head-on. LLMs can't be conscious--they are literally just matrix multiplication. But if LLMs can act like humans and not be conscious, then maybe humans aren't really conscious either.

View on HN

>>> If we were to somehow come to believe that consciousness is just an illusion--no more than a biochemical phenomenon, and no more special than transistor switching.

I believe this and yet I personally seek to inflict as little horror as possible, and am moderately restrained in the amount of force I believe the state should use.

> Literally our entire civilization is built on the idea that humans have qualia and therefore causing harm to humans is often unethical.

Here's an alternative angle that doesn't lead to dystopia:

Humans generate knowledge. Causing harm to knowledge is unethical. Broad human rights protect humans as knowledge generators (now or in the future or in potential) without going through the fraught process of arguing about each one.

Consciousness and qualia don't have to come into it.

Knowledge? That's entirely an anthropomorphic point-of-view issue. E.g. The impact of every drop of rain ever, is 'recorded' in the water table. All that data is lost, quintillions of bits of information, every day.

We don't harm humans because we are human, and value our kind. That's about it.

While I’m strongly sympathetic to your argument (or my paraphrase of it) that “I experience pain, so I’m willing to venture other humans do too, and so absent more knowledge than that, I should embrace empathy as far as I’m able”,

There is also a fairly rigorous bunch of work coming from David Deutsch and Chiara Marletto (his apparent intellectual heir, and seeming more than equal to carrying forward the legacy of thought) about how to talk about the “open ended generation of explanations with sufficient and increasing explanatory power” as more than hand-waiving.

They’re serious people doing serious work.

Knowledge is too broad a concept to protect all of it. Taking it to the extreme would probably mean that all output of /dev/random should be treasured in multiple backups, to avoid any bits to die.

It's straightforward to reject random noise as having no information content.

Not at all. If you had rejected the pretty much random processes in the primordial soup that eventually led to life (and then knowledge) from the start, then where would we be?

I think that it is almost impossible to determine randomness in an ongoing process.

Biological life itself embodies some knowledge. It uses trial-and-error evolution instead of thinking, but it arrives at solutions. Sometimes it arrives at the same solutions from different directions. This is different from randomness and the arbitrary results of random processes.

How will a system know all this initially? It seems this requires quite a competent judge and a lot of data.

The idea of knowledge being something to protect is interesting, but I don't see a practical way of accomplishing it.

Do our best? That's all we ever do when it comes to morals anyway. And I'm intentionally implying that we already value and protect knowledge, under a variety of different names.

Having said that, I don't really believe in one moral value to rule them all, I just think knowledge is a big one and useful for cutting through gordian knots.

It tends to return to feelings a lot anyway, since upsetting people isn't good for the growth of knowledge.

Aha, I think I'm starting to understand your position, which is obviously more nuanced than I initially thought.

But is knowledge such a big one? I suppose that the vast majority of the world population is not really helping to push the boundaries of knowledge forward. Unless you see gossip about celebrities as valuable knowledge as well.

> But if LLMs can act like humans and not be conscious, then maybe humans aren't really conscious either.

Isn't this like saying "if the man in the Chinese room can act like he understands Chinese and not understand Chinese, then maybe nobody understands Chinese."?

Yes. And that is one of the reasons why Chinese room thought experiment is interesting.

We know how a Chinese-understanding human would respond -- they respond exactly like the room does -- but we don't know if he actually understanding anything.

If you understood Chinese, you probably wouldn't have any doubt about whether you understood Chinese or not. And it seems incredibly strange to think that only you understand Chinese and nobody else does.

Why does ethics need to be based on consciousness anyway? Can't you just stipulate your baseline requirements and then enforce them by any means necessary? Are the aesthetics not "cosmopolitan" enough?

Reply on news.ycombinator.com