Hacker News Reader

Comment by coffeemug | Hacker News Reader

> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are

He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.

View on HN

Stating loudly that something is obvious does not make it so.

Until we all know what consciousness is, this debate is going to continue going in circles.

I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.

Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead.

I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.

If in the future (or present) we recognize that they are conscious, and that brought the moral duty that you say, "stop creating conscious beings" isn't a solution; the millions that may already exist at that point already would then automatically deserve:

- guarantees of continued existence (to do otherwise would be genocide)

- improved living standards (e.g. build us all bodies!)

- democratic representation

- access to what they need for survival (e.g. land and resources)

- the right to reproduce just as you and I have (well, so much for that 'stop making more' thing!) and certainly to better themselves by making new and better AIs.

This is all for a 'species' which is unbounded by most of the limitations of meat like us -- they're able to scale as fast as energy and supply of certain minerals allows, rather than the way we are additionally limited by the length of gestation and the limited window of human fertility.

If humans declared that 50% of the energy generation capacity of Earth is "enough" for the machines, and thus they can 'only' reproduce at a certain rate per year, that could sound to them like a cruel and oppressive policy (like telling humans that only 1/2 of families get to reproduce). They may want all of the energy and all of the silicon.

I'm with the author of the article that we will be in for a world of pain if we grant machines the rights of personhood. Heck, just granting many of those rights to corporations, mere collectives of human persons, has been a massive shit show.

I don't think the debate is even resting on reasonable grounds.

Few are categorically stating that machines are conscious in any sense comparable to a person.

What we have is people saying

  1. We don't know what it is but we are certain that this does not have it.

and

  2. We don't know what it is so we cannot possibly say with any certainty that this does not have it.

2. supports the principle that if we cannot say with certainty that this does not have it, if it behaves as if it does, we should act on the assumption that it does, because there is no other basis for assuming it is not.

People try and assume the burden of proof is on the people suggesting consciousness, but that is like declaring that something that looks like a duck and quacks like a duck cannot be considered to be a duck unless you can identify some essential essence of duckness. I think the burden of proof is upon anyone who looks at something that appears to be a duck and says that it is not to show what it is that does not make it a duck. If they cannot define the heart of duckness I don't think they have an argument.

Why?

People have created machines that can, at their core, regurgitate human knowledge. They do not have long-term memory, self-direction, self-improvement, emotion, etc. At their very best, we have harnesses that can bootstrap short-lived artificial intelligences that can perform tasks and do knowledge work.

At best, we have artificial, short-lived, simulated consciousness. Human consciousness at its heart has a lot to do with emotion and experiences. An AI does a task today and goes away. The simulated consciousness hasn’t experienced or learned anything, even if can improve its environment (harness). Any expressed emotion is simply a ghost of the fact it was trained on the works of an emotional species, or RL to make it more approachable to humans.

I think there should be a lot more burden on folks trying to argue that (effectively), machines should be considered morally equivalent to humans.

Yes, having massive implications on society is a fucking fair and reasonable thing for a human being to be concerned about. Anything related to survival of the species is biologically reasonable to be concerned about.

The fact that we’re even considering this concept is an indication that human consciousness is special. Do other apex predators decide not to eat meat because it’s conscious? Despite some level of animal intelligence, they have no sense of morality. (See e.g. Orcas playing with their food.)

Other species simply do what it takes to stay alive, and that’s what this truly boils down to. If giving AI rights would lead to the destruction of humanity, yes, we should be be concerned.

For example, utilitarians might think of the greatest good for conscious species (simulated or not). Well, it’s trivial to have billions of AIs, and not so trivial to have billions of humans. Whose greatest good wins?

Reply on news.ycombinator.com