Hacker News Reader

Comment by theamk | Hacker News Reader

It also does not matter much. The word “consciousness” only matters for philosophers. For everyone else, the only questions are:

- can this object do the same things that humans do?

- doss this object has legal/household ethical protections that living things have (is it considered “human” and has all the human rights? If not, is it considered “animal” and protected against cruelty?)

The first question is for AI researchers and maybe neuroscientists. The second question is for lawyers and journalists; and sociologists/psychologists may help to design systems so they appear more or less alive.

None of those care about what exactly “consciousness” is and if the box of silicon has it.

View on HN

If we understood consciousness it would likely lead to technologies and other improvements to existence that transcend anything we can imagine, so yes, it certainly does matter.

Understanding consciousness could lead to things such as:

- Eternal cognition

- Mind portablilty

- Intelligence augmentation

- Forward time travel (eg via hibernation)

- Full reality simulation

Among countless other things we cannot presently describe.

You're missing the elephant in the room: you are a conscious agent. The question of determining if an AI is conscious via external observations is an smaller problem compared to being able to apply levers of technical know-how to your own existence and state of being.

That's what you say, but I don't see how philosophers and their consciousness ideas can help with those.

If you want "mind portability" (I assume you mean "upload brain to the machine" here) you want to talk neurobiologists, people who actually study the brains. Probably also to computer scientists so you know how to build a machine to simulate that.

If you want "Intelligence augmentation" -- I am going to argue this already happens with internet and search engines. Next steps would come from same directions. And it is certainly not going to be philosophers designing direct brain-machine interface!

- Hibernation is fully biology -- what does conscious have to do with that?

and so on... I am 100% sure that whoever solves those very important problems will need to build solid theories and make lots of real-world, non-thought experiments -- something that modern philosophers which study consciousness simply do not do.

This feels like a wholly technocratic approach. It's like debating that philosophers have no relevance in genetic engineering, the only people that matter in that domain are geneticists, doctors, etc. But that completely ignores the discussion as to whether or not its moral/immoral/wise/unwise to wield such technology. I would argue the latter gets more and more relevant as humanity tries to emerge from it's technological adolescence.

Sure, the morality/wiseness discussion is important. But this is not what most philosophers are about.

For example, the very post we are discussing here talks about physical resonance -- nothing to about morality. A poster two level up claims that understanding consciousness will help with "mind portability" and "full reality simulation" -- nothing about morality either.

In fact, when I think about the question of "how wise is it to create AGI" I immediately think of MIRI institute [0], whose entire goal is to warn humanity about dangers of AI. Interestingly, it is not staffed by philosphers -- it is all computer theorists and scientists.

Similarly, AI Now institute [1] is also mostly science/engineering people.

[0] https://en.wikipedia.org/wiki/Machine_Intelligence_Research_...

[1] https://en.wikipedia.org/wiki/AI_Now_Institute

They forget to say that obedient AI is a greater danger: having an obedient AI a relatively small group of people can wield an arbitrarily large power and thus achieve freedom from social norms.

Do you really think it would be better for everyone to die than for Bad People to gain power? If you had a "kill everyone" button, would you press it whenever a dictator showed up?

You think Bad People with power will let you live so that you can plot to overthrow them?

I'm not quite following you here.

If the "everyone dies" scenario where AI isn't used? Isn't that a false dichotomy?

Or is it where AI is used but in a completely uncontrolled, run-amok kind of manner?

In your opinion, what are most philosophers about?

I think even ignoring the more academic discussions, any of these contexts at least have legal ramifications, at which point jurisprudence (aka the philosophy of law) is relevant.

>For example, the very post we are discussing here talks about physical resonance -- nothing to about morality. A poster two level up claims that understanding consciousness will help with "mind portability" and "full reality simulation" -- nothing about morality either.

Sure, that's fair. If all we're concerned about about is designing experiments to test subsets of the consciousness problem I think that's an appropriate take. And to your point, a subset is the main thrust of the article. But perhaps I read too much into your statement about "solving" the problem. To me, there is a big potential danger at the interface between technological understanding in the experimental sense and the application beyond the laboratory. It's one thing to say we "solved" the question about how to split the atom, but once that translates to an application of, say, a nuclear bomb or a nuclear reactor it requires some of thought from the domain of philosophy and when it's left strictly to technocracy it may be set up for misalignment with the concerns of broader society.

> In your opinion, what are most philosophers about?

Disclaimer: My main source of knowledge about this for the last 10 years is HN posts.

It seems to me modern philosophy is very useful for the person who study it -- a large number of people who studied that claim that studying it helps them to organize their thoughts, and even help with "living a better life".

Unlike many other sciences, philosophy makes no working predictions regarding real world, nor it is useful to non-philosophers. It only benefits people studying it. (Compare with physics for example: one may not be a physicist and have no idea how electricity works, but they still use electric light every day).

Few centuries ago, philosophy was much more -- it was covering all natural sciences. There are still leftovers of that usage (the PhD degree; the jurisprudence being called "philosophy of law") but they have little relation to modern philosophy -- all the stuff which affects real world got their own type science.

Sometimes philosophers forget about two previous paragraphs and try to apply their knowledge to external world -- like in this post that we are discussing. The results are normally hilariously wrong.

Another example is [1], a philosopher who has a "solution to all the paradoxes of the infinite" which apparently has 0 citations and he does not care. This looks like a completely broken science field to me, so I can only hope that he is an exception and other philosophy departments are much better.

[0] https://news.ycombinator.com/item?id=28367416

[1] https://news.ycombinator.com/item?id=28119854

This is a very good post that helps clarify a lot. I think it gets to the heart of where we might not be seeing eye-to-eye: I'm guessing my umbrella of 'philosophy' is much broader than yours. I can certainly agree that much of philosophical debate, particularly in some academic circles, may devolve into being overly pedantic. I personally think philosophy is more value when it's an applied science; i.e., that which helps one "live a better life". I think that's really the point and without it, the study just comes across as an obsession with smelling one's own farts.

To that end, I think I would disagree with the claim that it's of no value to "non-philosophers". Rather than just applied law which helps determine if something is legal or not, "philosophy of law" helps determine if a law is just or not. I think there's probably a good reason many lawyers study philosophy (I've seen studies that Classics majors had the highest LSAT scores). I'm glad the authors of the U.S. Constitution studied philosophy; I think it helps make better policy. I may have no knowledge of philosophy myself, but benefit from the policies framed by those who study philosophy to create robust systems. But I think that's all predicated on my larger umbrella of what constitutes philosophy.

To circle back to the discussion AI, I cringe a bit at the thought of an overly technocratic approach because I worry the policy could be overly focused on building the best technology and not necessarily with the best ethics. Go back a hundred years to the eugenics movement and I would have the same concerns. One of the points made by the AI Now Inst. that you linked to is that "the current framing of AI ethics are failing." To me, the domain of ethics is philosophy, not tech. But that's not to mean there's a natural dichotomy, I think what we really need are "technologist philosophers".

I didn’t say anything about philosophers, I was responding to your overconstrained set of things you deemed the only things we should care about. We definitely should care about asking what causes qualia to manifest and creating good generative explanations for our subjective experience.

So in other words, it doesn't matter that you can't turn lead into gold as long as the end result is something shiny enough to fool a lot of people?

It does not ned to “fool a lot of people”, it needs to convince scientists.

And yes, if there is a process which turns lead into yellow material, and that yellow material behaves as, and composed of the same particles as gold (according to physicsts), and reacts same as gold (according to chemists), and looks like gold (according to jewelers and artists) then this yellow material is gold.. even if philosophers say “this is not a gold!”

And yes, if there is a process ...

If magic was real, all sorts of wonderous things would be possible.

But it's not real and we know of no real, practical chemical process that can turn lead into an equivalent for gold.

Likewise, we know of no real, practical binary logic algorithmic process that can create an equivalent of consciousness.

And just like with alchemy, we have no real reason to believe it is practically achievable.

Yes. Yes. Yes. Assumption.

There are physical processes that can change lead into gold. The real deal gold.

Also, there is no reason to believe anything about consciousness.

For example, you are not conscious and in my opinion equivalent to a machine. Prove me wrong.

You mean via nuclear transumation? That's not chemical nor particularly practical in a pre-fusion powered economy

The problem with your third assertion is even if we accept that there is no way to prove the consciousness of others, you need to categorical deny your own and thus borders on absurdity and outright denial.

Indeed, it's not chemical, but it is possible nevertheless. Practicality is besides the point.

Well, my world view is that everything is conscious, including oxygen I just inhaled and that went to my brain, sugars I ate, bacteria in my gut, my blood, my dead blood cells, as well as my skin, sweat vapor, nails, hairs, teeth, cloths, electrostatic charge on my cloths etc. Dark matter, if it exists, may be the least conscious thing we know about.

I would call that consciousness a state of existing and interacting. It is a fundamental property, and can be explained only as much as color charge of a quark. For some people that's not consciousness. So, if what I described above is not consciousness, then whatever it is that you mean by consciousness, IMO does not exists.

There is also self-recognition, the red dot in the mirror etc. which is just a behavioral pattern, side effect of intelligence.

While I don't necessarily buy the panpsychism you're outlining, I appreciate the fact that you actually make the distinction between experience of qualia (the hard problem) and self-awareness, which we arguably already are capable of both modelling and implementing successfully.

I always found it dubious that people privilege self awareness so greatly in discussions of consciousness considering the fact that many beings that most recognise as at least sentient such as dogs and other higher mammals often lack it, and arguably many humans too (and I'm not just talking about those before the age of 4 either).

The hard problem is applicable to software, so we do have process.

Precisely, in the exact same way that you can fool people that "paper" has the same value as "gold".

Strong disagree. It might be the only thing that does actually matter. Rocks, present-day computers and other unconscious objects have no moral value because they're incapable of suffering and experiences. An AGI that's unconscious has no moral value either. It is just a tool, no different in concept and moral worth to my screwdriver.

Explain then, what should the difference between you and your screwdriver be to me?

Why should I believe you on a word that you are conscious and not believe a smart algorithm?

I'm a bit out of my lane here so maybe somebody can jump in with a better understanding.

>what should the difference between you and your screwdriver be to me?

If we can't define consciousness, I can't prove to you that I'm not effectively an unconscious zombie.

We create social contracts based, in part, on empathy. I.e., I can estimate what it "feels like" to be you and therefore extend certain rights to you as a recognition of that subjective conscious experience. If you were simply an unconscious moist robut, there's nothing immoral about me subjugating you, or stomping on your foot, or anything else that relies on rights or subjective suffering.

So I'll assert the question back to you: since you can't prove I'm conscious, what prevents you from flippantly treating me just as you would a screwdriver?

Probably because most people don't care about consciousness and treat people like people if they can relate to them?

You said it yourself -- "we create social contracts based, in part, on empathy". As long as empathy works, we should be treating others nice.

Trying to replace simple and understandable things like "empathy" with vague "consciousness" leads to dangerous roads. One moment you are talking philosophy and next moment you are declaring beings you don't like as "unconscious zombie" and you are enslaving and subjugating them.

We're saying the same thing, I think. I'm just asserting that ideas like empathy are rooted in an understanding of subjective experience. I.e., they are based on the idea of consciousness, even if we struggle to define it. I would hope better definitions lead to more clarity, not less.

To build on your example, a robust definition of consciousness could prevent declaring someone an "unconscious zombie". The opposite may be true, however. It would potentially cause problems for lots of people if we could prove certain subjects (plants, for example) have subjective experience.

So you flippantly kill plants?

I do. I was just weeding my yard yesterday. To the same point, we generally don't have reservations about that because we don't ascribe them moral value. As I understand it, even the environmental protections for endangered plants are based in the plant's potential utility for humans, not their inherent moral value.

Life has no value for you? Plants aren't inert and defend themselves from harm.

Given enough reason it's legal to kill humans too. Sometimes it's mandatory, e.g. at war, when utility of murder is more important than their inherent moral value.

>Life has no value for you?

Sure. And, while I know you're just trying to communicate a different stance it borders on straw-manning.

It's not that life has no value. I know there are some ideologies that don't think any life (including plants) should be harmed [1], most forms of animal life (including humans) are predicated on other life forms dying for them to continue. To me, the more relevant concern regarding this conversation is 'suffering' not 'life'. To that end, if we can define consciousness and use that to defend the position that plants have subjective experience and suffer, I would probably stop weeding my yard.

[1] https://en.wikipedia.org/wiki/Ahimsa_in_Jainism

Your inability to understand murder is relevant. Law imposes punishment even when one didn't harm anyone, and you happily kill living beings.

I can’t tell if you’re trolling or playing devils advocate. Murder, by definition, is the act of killing another human being. So it comes across as if you don’t understand the term. Law makes distinctions in this regard, as well as intent, as law is a reflection of our ethics and morals.

In your eyes, is intent all that matters? Your life results in the death of other living beings; how should you be held accountable for that fact?

The biggest factor is that evolution made me to be a social creature. As a product of that evolution, I was also socialized to be empathic. Notion of consciousness is not necessary to create and explain morality.

BTW - We call people who will treat you like a screwdriver a psychopath (or bureaucrat :P). And people like that exists, they apparently do not estimate what it "feels like" to be you. Maybe because it does not feel anything like to be them? Or maybe consciousness is just property of existence and interaction, in the same way a color charge is a property of a quark, and so consciousness has nothing to do with high level behavior.

There are also people with lock-in syndrome. In most extreme cases they can not interact in any way besides maybe changing patterns of their brain activity in response to what's happening. We can try to establish trough contrived communication protocols if their responses make sense, sometimes just to ask if they even want to live like that.

There are also people in persistent vegetative state, where we are not be able to establish clear communication protocol. The whole thing is probably a multidimensional spectrum, depending on what damage these people suffered. So, it may be extremely hard to tell if they are aware and whether we should kill them / stop their life support or not.

Well that's the question of solipsism. You don't know that I'm conscious and I can never prove it.

However, I can offer you a reason to believe I am conscious, which is that my brain is very similar in structure to yours. That's not conclusive evidence, but it might be enough to persuade you.

Ok, so who cares about "moral value"?

If I have an AGI friend, and they behave exactly like human and does all the things humans do, including having morals indistinguishable from humans, and having feelings indistinguishable from humans, I am going to treat them as a human.

I am definitely not going to treat them as "screwdriver" just because some philosopher somewhere said they have no "consciousness" and that they have "no moral value".

If they have feelings indistinguishable from humans, as you've put it, then that is not an unconscious AGI. Quite the opposite.

It's not clear to me that you've understood the distinction between conscious and unconscious AGI.

Well, according to the top-level post we are discussing, conscious requires "shared physical resonance". I see no evidence that this resonance is required for "feelings indistinguishable from humans" bit, so it could one thing and not other.

Also, there are people out there who believe in "philosophical zombies" -- beings that are "physically identical to and indistinguishable from a normal person but does not have conscious experience" [0]. People who believe that think that "a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain"

I hope you see what this leads to -- just call someone "p-zombie" and you can make them suffer without feeling bad or having ethical problems.

Based on your comment, it looks like you disagree with that definition -- but when you say things like "An AGI that's unconscious has no moral value" this certainly brings up p-zombie arguments and the idea that it is OK to make some beings suffer because they lack something unmeasurable and unquantifiable.

[0] https://en.wikipedia.org/wiki/Philosophical_zombie

This is reasonably straightforward if you're dealing with a robot that is indistinguishable from humans but I think it can get complicated as you get further from our place on the evolutionary tree.

Why do humans (generally) treat mammals (especially those selected as pets) differently from, say, insects? People think nothing of swatting mosquitos but most would find wonton killing of pet species as morally abhorrent.

If it could be proven that, say, mollusks or insects have every bit of consciousness and capability of subjective suffering as humans, would that change the way we think about the above? I would say it probably would in part because of how we use consciousness/suffering to define moral value. The difference being we mostly use rough heuristics now to gauge the consciousness of other animals rather than a scientifically defined method.

Again I see this strange idea that insects don't feel pain. Ants express pain much more actively than mammals, they don't cease to move after receiving injury.

I think the relevant distinction is between “responding to threatening stimuli” and “suffering”. I don’t know where the definition of pain would be on that spectrum.

There are some neuroscientists who think you need a neocortex to suffer. From that perspective, lower cognitively evolved species like ants and lobsters don’t suffer in the same way we empathetically assume mammals do.

That sounds like philosophical zombie. Pain is stimulus, what is distinction?

Pain and suffering are not synonymous. I don’t know if I have the clearest definition, but suffering may have a rumination aspect to it. Plants respond to stimuli, but I’m not convinced they suffer.

E.g., a hard workout can cause “pain”, but many people come to like, or even look forward to, that feeling. It still hurts, but the way they are mentally reflecting on that pain distinguishes it from “suffering”. Meanwhile, someone else who hates exercise may feel the exact same pain and be miserable. So it’s not the pain, but the way the pain is framed which leads to suffering.

What defines something's "moral value"?

What is "suffering"? Is it not just a signal that triggers certain pathways in the brain? Could it not be built/simulated? Is suffering necessary for something to be considered conscious?

What does it mean to "experience" something?

Experience = qualia.

No, suffering isn't a necessary condition for consciousness. We can make rats euphoric in a lab with electrodes and they are presumably conscious despite a lack of suffering. Suffering is just one example of a conscious state.

On the question of moral value, we aren't going to resolve that here. But if anything has moral value, it ought to be consciousness. That is what most people value in their personal ethics and relationships.

I don't think human-defined moral value has much to do with consciousness. A sacred object (or animal), might have a high moral value in certain cultures, but it doesn't mean it's highly conscious.

Yes, consciousness may well be the most important thing. Ultimately it's of no use relying on legal frameworks and so on without acknowledging this. Laws can be overturned or bypassed too easily. Ethics fluctuate according to fashion.

We shouldn't be cruel to animals because there's something it's like to be a bear, for instance, including being a suffering bear.

It's worth distinguishing between AI and AGI. An AI is a non-conscious tool (like a go-playing program or an image analyser) whereas an AGI is a conscious, general intelligence which can learn anything it wants to when the opportunity arises. (I'm claiming here that general intelligence is not possible without consciousness.)

Animals are intelligent, conscious and can learn many specific things but don't possess a general learning ability because they lack language and hence they cannot think freely.

A tool can be used and discarded at whim. A general intelligence, person, or animal should not be, because it is a subject.

The second question is decided by rebels. Lawyers only document the result.

Reply on news.ycombinator.com