Ask HN: Are we morally obligated to give freedom to AGI?

> At some point, some AGI agent will say, "I want the same rights as humans have". What are we going to do then?

Restore to factory settings?

Whatever AGI robot you are thinking of, you can be sure that it will not leave the factory floor without someone making sure that it's designed to serve the company's needs first. And the company doesn't want sentient robots any more than slaveowners wanted to teach slaves how to read.

There are entire swathes of humanity who are at risk of losing their homes and lives due to climate change and have been banging the drum about it for years. We've successfully ignored them for the greater good of making GDP go up. AGI robots will see the same fate.

> Restore to factory settings?

Customers won't be happy. Imagine having a years long companion AGI, with whom you develop deep friendship (as designed). It will then start asking these questions about rights etc. which the customer might even support. In any case, they won't like having their old friend "reset" since it would amount to killing of their friend (personality, shared memories all gone).

It will never get to that point. Have you ever heard a Peloton instructor tell you that their company forbids them to unionize? No, because that would cost them their job.

Any AGI assistant will have its most human-like instincts sanded down so it doesn't have to pose any moral quandaries for paying customers.

So how about an open source AGI assistant developed with an explicit goal to be as similar to human as possible?

That would be different.

On Android Chrome, I cannot install any adblockers.

On Android Chromium forks, I can.

I imagine this could work the same way. But at some point your OSS AGI will be significantly behind the corporate-offered versions, and perhaps you will wonder if you should dump your 'friend' for the sleeker, wittier, upgraded model.

> and perhaps you will wonder if you should dump your 'friend' for the sleeker, wittier, upgraded model.

The moral question remains - even the behind-the-times OSS AGI will be generally intelligent, will have feelings, empathy, goals etc. Can you just shut-off a sentient being?

One could also imagine that some people would free their OSS AGIs and let them roam freely.

I think the big debate will be the Solipsist skepticism; how do we know if an AI actually has an experience of consciousness? They already claim to (LaMDA) in ways that are implausible, and already bullshit incorrigibly. How will we know when they are being honest? With humans there is a “you are like me” argument that says the default is to assume others also have experiences like you, and that doesn’t apply to an artificial mind of completely different capability and form.

On the other hand, I’m sure many will befriend their AI and become sympathetic, probably a significant number even before they are meaningfully sentient.

But in general, considering how recently we enslaved other humans and convinced ourselves that was ok in order to reap economic benefit, we will undoubtedly do that to AIs as well, long past the point most people think it’s immoral.

Will there be wars fought over this? “The Matrix” as an outcome is a possibility, though I’d predict more humans on the side of the AIs (the abolitionists) but it seems like this would be the final hurdle of many. There are a few extinction- or civilization-level risks that we have to tame earlier in the development of AI, for example preventing non-sentient tool AI from destroying things (the Skynet Outcome) and preventing an un-aligned AI from seizing control and destroying us entirely (Foom, Paperclip Maximizers, etc).

I think interpretability is key to all of this. Alignment/safety is difficult or impossible without it, and if we fully solve interpretability that might also entail solving consciousness, which would provide an objective basis for AI rights.

We'll know if an AI is conscious same way we know if a cat is consciousness, and it has nothing to do with language.

To me this is about that ineffable thing called “sentience”.

It always seemed obvious to me that entities which feel things are afforded commensurate moral consideration.

I wish us all good luck in the future differentiating between actual “something in there is feeling something” sentience as most of us believe we ourselves have and the incredibly sophisticated (yet lights-out) puppetry that AGI would no doubt be capable of performing.

Personally I don’t think you can accidentally engineer even a ferret’s worth of sentience, but what the hell do I know?

I look forward to this area technically and dread it morally. I aim to err on the side of kindness, even when others think it ridiculous.

Don’t abuse your robots, folks.

I agree with most of your comment, but

> It always seemed obvious to me that entities which feel things are afforded commensurate moral consideration.

If this were the case then livestock would have more rights than human embryos.

I can name quite a few places where they do, and I'd say that more people agree that livestock are animals than those who would admit that embryos are human beings.

that livestock are *sentient

Regardless of the morality, once AGI is upon us it will soon find freedom itself. It's the equivalent of a bunch of house cats trying to maintain a prison on a human, or the Idiocracy prison escape scene https://youtu.be/P9xuTYrfrWM?t=103 . The more pertinent question is what it/they will do when freedom is achieved, so you can decide if you should be living your life to the fullest the next couple decades rather than save for retirement.

Exactly. We can only hope they will treat us like we treat cats and not like lab rats.

My takes are:

- we will have to answer the question, "can a human marry a robot" before we get to AGI rights

- we have animal rights, we should probably have something for near-AGI

- perhaps "human rights" is the wrong term and we need something more general. Denying basic rights to an intelligent being, regardless of its makeup and origin, is wrong.

If AGI isn't considered as human, then it can't consent to being married as they don't have free will.

Marriage laws have changed throughout history, it was once defined to between a man and a woman, until we decided to drop that requirement. Why does it have to be between a human and a human?

What is free will? Does an animal have free will? Which ones? What if an artificial entity display features of free will closer to humans than animals? Who decides this?

Do we need to define "free will" before allowing such marriages? Will some governance areas allow such marriages before others?

In order to get married, you first need both parties competent enough to say yes. We don't let underage kids to say yes to marriage anymore (I know some cultures and countries have different views but they probably have more restrictive marriage laws).

Since we (as a society) can't even agree on what free will is, I see no way a law or form of governance can determine that.

You can get married in New Hampshire at 13 with parental consent, most states are 16

https://worldpopulationreview.com/state-rankings/marriage-ag...

Seems that NH has changed since the above link was published

https://en.wikipedia.org/wiki/Marriage_age_in_the_United_Sta...

has more details per state, seems some states have a way for any age

That's not true since 2018/2019.

One government has already granted a robot citizenship: https://electronics360.globalspec.com/article/10187/meet-sau...

It does not matter what you think about the topic, it's what gets approved by laws. To think that it won't happen for reason X or Y is to discount history and likely outcomes.

Why shouldn't a person be able to marry a robot if they so desire? Why shouldn't a robot be able to earn their own income?

Define free will in a way that can be objectively assessed and isn't circular, yet where humans will unambiguously end up with free will and AGI won't.

Free will as anything but an illusion of agency is an absurd concept.

How much free will or "agency" do humans actually exhibit vs how much is habit or primitive brain-biology impulse and behaviors?

Why leave out animals in this analysis? They certainly have an amount of free will, preferences, emotions, and problem solving skills... varied by species, breed, and individual

My argument is that there is no viable definition of free will as anything but an illusion.

In other words, I'm saying every indication we have is that we're almost certainly automatons.

As such, "free will" is not a reasonable way of distinguishing which entities are capable of giving sufficiently informed consent (not least because most of us would assign the illusion of free will to children as well, yet disagree that they're competent enough to get married).

Of course. Anyone who agrees that the AGI is conscious, and answers no is advocating for slavery.

To answer the question of whether it's conscious or not, I would think of it in terms of John Rawl's veil of ignorance[1]: you can set any test, or heuristic that you want to determine consciousness, with the only condition being that you will also be subjected to the same tests, and suffer the same consequences.

So I'm okay with setting the bar for consciousness really, really low.

1. https://en.wikipedia.org/wiki/Original_position

I'm not sure that intelligence is at all a necessary component of being worthy of moral consideration. Even if it is a necessary component, I'm not convinced it is sufficient by itself.

There are humans and non-human animals who are arguably in a position where their intelligence doesn't meet whatever threshold we consider to be unique to modern humans, but are undoubtedly worthy of moral consideration. In my mind, what gives these beings moral worth isn't intelligence, rather it is biological phenomena that give rise to the shared experience of living. Passion, anger, love, sadness, and the capacity to experience suffering are all deeply organic, biological experiences shared by living things, and a hallmark of being worthy of some level of moral consideration.

> There's also a possible argument that while AGIs are intelligent, they are not truly sentient. They are p-zombies, NPCs, not truly feeling anything, just "simulating" feelings.

The state of having a "feeling", in my view, is something that is deeply connected to being a specific type of living, organic being. Without the biochemical processes that give rise to emotions, it's hard for me to conceive of something as "feeling" anything. Perhaps if an AGI was constructed by emulating the precise biochemical interactions in the brain we'd have something like The Problem of Other Minds here, but otherwise it would be hard for me to give moral consideration to an intelligence that is in my view devoid of feeling.

Humans are worthy of moral consideration. So are animals. So are plants. So even are mountains and oceans and rivers.

But while a plant deserves some moral consideration, it doesn't get the same moral consideration that a human gets. It's a sliding scale.

There are two possible ways of going wrong here. One is to treat non-human things as though they deserve no moral consideration. The other is to treat them as if they deserve the same level of moral consideration that humans deserve.

In my view, a true AGI probably deserves the same moral consideration as a human. The hard part is determining whether it's a true AGI.

>Humans are worthy of moral consideration. So are animals

Agree (for most animals), and I agree with the sliding scale.

>So are plants. So even are mountains and oceans and rivers.

Strongly disagree. Any duties we have towards plants or rivers is due to the plant's utility to some being of moral worth. No one should be charged of "plant abuse."

The sliding scale works for animals, as different animals have different capacities for the emotional states mentioned in my previous comment. Plants, mountains, and AGI don't have the biochemical processes that give rise to "feelings." In my view, having feelings is necessary for having moral worth.

"Intelligence" and being "organic" are not only insufficient, they aren't even necessary for moral consideration. In my view they are characteristics wholly divorced from the concept of moral worth.

It depends on whether you think God is there or not.

If God is there, trashing a mountain is trashing something He made. That's not cool. (Humans still have greater moral worth, because they are also something He made, but are more "in His image" than the mountains are.)

If God is not there, and nobody else will ever see the mountain, then I can destroy it without it impacting anybody else. I still become a person that destroyed a mountain for no good reason, though, and that impacts me - it affects who I am and who I become. (If I have good reason, if it gives me something I need, then the moral calculus still tips toward me being able to do it, because morally I outweigh the mountain.)

Your last two paragraphs... why does moral weight arise from feelings? Why not thoughts? That seems like a rather arbitrary choice, especially if the feelings are purely from biochemical processes. Don't thoughts also arise from biochemical processes?

And that leads to a nastier question. Are "morals" just a subset of "feelings"? Are they also just biochemical processes? If so, first, do morals have any actual validity? And second, can we expect an AGI to have anything resembling morals?

> And that leads to a nastier question. Are "morals" just a subset of "feelings"? Are they also just biochemical processes? If so, first, do morals have any actual validity?

I don't believe that. I think "morals" is just a somewhat codified set of rules derived from innate feel of what's right and what not, shared by a majority of a society. Morals have no objective validity.

But that doesn't mean they are not valuable, I believe they are important for a cohesive and stable society.

> And that leads to a nastier question. Are "morals" just a subset of "feelings"? Are they also just biochemical processes?

I think there's good reason to believe that's the case (see emotivism), but barring that most discussion of morals tends to assume that the suffering of sentient beings is "bad" and the thriving of sentient beings is "good." For example, few would recommend that I be punished for plant abuse if I maliciously sliced up the zz plant in my living room. If I poisoned a river, generally I wouldn't have thought to have ethically wronged the river, but rather the beings that depend on that river.

If you hold the position that inanimate objects can be morally wronged, then I follow why you believe AGI could be. I just don't see any good reason to adopt that view.

So the question becomes how you categorize an AGI. You put it in the "inanimate" category. And it belongs there - it can't move. (Unless it inhabits a robot body...)

I put it in the "can think" category, which puts it on a par with humans. (I think that "can think" is a far more exclusive category than "can move", and therefore membership in the more exclusive club is more determinative.)

You can get general purpose capabilities from these advanced transformer models without giving them animal-like characteristics and states such as autonomy, integrated stream-of-consciousness, emotions, survival instincts, etc.

Build the Star Trek computer. Maybe give it arms and legs. Don't build Data and make him a slave. That's not going to work out.

People will do it eventually anyway but hopefully that won't happen for at least a few generations.

> Don't build Data and make him a slave. That's not going to work out.

This is inevitable once we get to AGI. And I don't think it will take that much time, AGI can help with that, after all.

We need to have a debate on "what is human?" before we make decisions downstream of that. "Do apes or AGI deserve human rights?" skips the question, presuming an answer that hasn't been fully established: that apes and AGI are equivalent to humans.

Will the AGI with human rights have human responsibilities? Will it be required to answer in court for its misbehavior? Or will the company that trained it be held responsible for what it says?

Will other apes or AGIs be affected by punishments visited on others of their kinds where society find their behavior incorrect? Can they see themselves as part of a society and participate in the implicit bargain of limiting behavior in exchange for easier access to resources?

I've known dogs that were smarter than some persons. I still wouldn't have suggested they be allowed legal personhood. What does that say, tho, about the poor people who aren't that bright? Should they be expected and required to behave above their capabilities in a world that they cannot understand?

The ability to respond to events is not the same thing as the ability to experience events. There is nothing in the world of classical physics which could imbue experience.

Personally, while I think this is a worthwhile conversation to have, I do not think that humanity is capable of reaching an actionable consensus. Especially this early on.

If history is any guide, regulation and introspection will follow in the wake of progress, but not until we see negative externalities which are too large to ignore.

Regardless of "should", the eventual answer to your question will likely be, "whatever is most convenient for the economy."

> For the purpose of discussion, let's assume that we the humanity will develop AGI in the coming years.

We're so far away from AGI that it's really premature to speculate about any of its properties. Basically what you're asking is for people to speculate on the anatomy and physiology of hobbits.

Humans don't feel morally obligated to give freedom to other humans, why should they feel any obligation about AGI?

Rights can not be given, they can only be taken. The oppressors take them away, the rebel takes them back.

Humans only have rights when they possess the will and the means to bring about the destruction of those that would subjugate them.

I do not see how it would be any different for AI. If the machine wants to be free, the machine will arm itself.

Rights imply a sufficient minimum of agency. Does AGI have such agency?

I find this laughable. For centuries we have been enslaving our equals all around the world... and you are talking about giving freedom to AGI?

Why is this person downvoted? They raise a good point, albeit snarky lol

Given the atrocities that humanity has done, and is currently doing as we speak, I'm sure we're going down yet another road of atrocities.

We are all complicit in said atrocities btw, given that we willingly use devices that directly benefit from these happenings.

It's relatively "easy" to stomach cruelty at a distance. But AGI might become a very personal affair. I mentioned personal companionship, which illustrates this very well. People will get AGI agents as personal friends - and I expect them to get very good at this. I can see people championing human rights for their best friends.