Comment by benreesman | Hacker News Reader

Comment by benreesman | Hacker News Reader

Parent Full thread

Is there some particular reason why this debate isn’t squarely in the spiritual/religious/personal/subjective building on campus and zero in the science building?

I mean, these are all fascinating questions in a “let’s smoke a joint and talk about the meaning of life” sense, I’ve had many such conversations (both with and without the joint) and enjoyed all of them that were of that tenor.

And I can see there being a building on campus between the other two where the topic is ethics and morality: how rigorous can we be about what constitutes acceptable behavior, compassionate behavior, empathetic behavior, kindness and decency. Those things seem much more amenable to some level of rigor: I certainly hope that some version of those ideas can be rigorous enough to admit a consensus, but that seems like a way more realistic goal than defining consciousness. It still poses hard questions: is it ok to eat animals? That’s controversial but seems at least in principle amenable to scientific study of apparent pain or suffering and strategies for minimizing or eliminating it entirely.

I have a sinking suspicion that the real definition of “conscious” is “seems a lot like me”.

I'm not sure if you're dunking on "spiritual/religious/personal/subjective" stuff, but the hard question of consciousness is, in some ways, the most important question of all. Far more important than "science" questions like "what is dark matter?"

Literally our entire civilization is built on the idea that humans have qualia and therefore causing harm to humans is often unethical. If we were to somehow come to believe that consciousness is just an illusion--no more than a biochemical phenomenon, and no more special than transistor switching--then we would usher in a dystopia in which any horror inflicted on people is potentially justifiable and even necessary.

The rise of LLMs is forcing us to confront this head-on. LLMs can't be conscious--they are literally just matrix multiplication. But if LLMs can act like humans and not be conscious, then maybe humans aren't really conscious either.

>>> If we were to somehow come to believe that consciousness is just an illusion--no more than a biochemical phenomenon, and no more special than transistor switching.

I believe this and yet I personally seek to inflict as little horror as possible, and am moderately restrained in the amount of force I believe the state should use.

> Literally our entire civilization is built on the idea that humans have qualia and therefore causing harm to humans is often unethical.

Here's an alternative angle that doesn't lead to dystopia:

Humans generate knowledge. Causing harm to knowledge is unethical. Broad human rights protect humans as knowledge generators (now or in the future or in potential) without going through the fraught process of arguing about each one.

Consciousness and qualia don't have to come into it.

Knowledge? That's entirely an anthropomorphic point-of-view issue. E.g. The impact of every drop of rain ever, is 'recorded' in the water table. All that data is lost, quintillions of bits of information, every day.

We don't harm humans because we are human, and value our kind. That's about it.

While I’m strongly sympathetic to your argument (or my paraphrase of it) that “I experience pain, so I’m willing to venture other humans do too, and so absent more knowledge than that, I should embrace empathy as far as I’m able”,

There is also a fairly rigorous bunch of work coming from David Deutsch and Chiara Marletto (his apparent intellectual heir, and seeming more than equal to carrying forward the legacy of thought) about how to talk about the “open ended generation of explanations with sufficient and increasing explanatory power” as more than hand-waiving.

They’re serious people doing serious work.

Knowledge is too broad a concept to protect all of it. Taking it to the extreme would probably mean that all output of /dev/random should be treasured in multiple backups, to avoid any bits to die.

It's straightforward to reject random noise as having no information content.

Not at all. If you had rejected the pretty much random processes in the primordial soup that eventually led to life (and then knowledge) from the start, then where would we be?

I think that it is almost impossible to determine randomness in an ongoing process.

Biological life itself embodies some knowledge. It uses trial-and-error evolution instead of thinking, but it arrives at solutions. Sometimes it arrives at the same solutions from different directions. This is different from randomness and the arbitrary results of random processes.

How will a system know all this initially? It seems this requires quite a competent judge and a lot of data.

The idea of knowledge being something to protect is interesting, but I don't see a practical way of accomplishing it.

Do our best? That's all we ever do when it comes to morals anyway. And I'm intentionally implying that we already value and protect knowledge, under a variety of different names.

Having said that, I don't really believe in one moral value to rule them all, I just think knowledge is a big one and useful for cutting through gordian knots.

It tends to return to feelings a lot anyway, since upsetting people isn't good for the growth of knowledge.

Aha, I think I'm starting to understand your position, which is obviously more nuanced than I initially thought.

But is knowledge such a big one? I suppose that the vast majority of the world population is not really helping to push the boundaries of knowledge forward. Unless you see gossip about celebrities as valuable knowledge as well.

> But if LLMs can act like humans and not be conscious, then maybe humans aren't really conscious either.

Isn't this like saying "if the man in the Chinese room can act like he understands Chinese and not understand Chinese, then maybe nobody understands Chinese."?

Yes. And that is one of the reasons why Chinese room thought experiment is interesting.

We know how a Chinese-understanding human would respond -- they respond exactly like the room does -- but we don't know if he actually understanding anything.

If you understood Chinese, you probably wouldn't have any doubt about whether you understood Chinese or not. And it seems incredibly strange to think that only you understand Chinese and nobody else does.

Why does ethics need to be based on consciousness anyway? Can't you just stipulate your baseline requirements and then enforce them by any means necessary? Are the aesthetics not "cosmopolitan" enough?

You might be being facetious, but just in case - academic philosophers generally aren't in the business of smoking joints while talking about the meaning of life, or anything like this (even without the joint). They're generally focused on making principled arguments for views, including views on consciousness.

And the debate is primarily a philosophical debate, not a scientific debate, if that's what you're asking.

And I'm not sure that ethics is particularly more "scientific" than philosophy of mind. There's a case to be made that scientific study of pain is relevant to the morality of eating animals, yes, but there's also a case to be made that science is relevant to consciousness, e.g. the science related to IIT. And in both cases, the science is relevant but doesn't even come close to solving the issue.

Science generally maintains silence in these sort of discussion in my experience, there isn't a lot it has to say and I don't think any of the facts are controversial. There aren't really any questions here about observable phenomenon. But scientists also enjoy philosophy and it is an easy topic to have an opinion on. And arguing is fun for its own sake, although some people seem to be motivated by fear of their perception of reality being challenged.

Reply on news.ycombinator.com