We must not grant AI agents legal personhood

Suppose I have a local LLM running on my PC. I have close and personal conversations with it. It keeps a chat history. I'm later accused, falsely,* of a crime. During the process of defending myself, should the prosecutor be entitled to a copy of my LLM's conversation history that was seized pursuant to a warrant? Or allowed to interrogate my LLM, again seized while executing a search warrant?

Under current US law, the answers are probably yes. Just as an accused's diary or notebook can be seized and introduced into evidence, an LLM's internal state seems like a paper/effect/document that would be subject to search and seizure. But a conversation with a local LLM also seems more like a conversation with a spouse, therapist, or attorney. There are good public-policy reasons to treat those conversations as privileged; we want people to be able to get good advice without fear of disclosure. Shouldn't those reasons apply to LLM conversations as well?

I'm not saying LLMs should have personal rights. But I can see a case for privileges for real humans arising from conversations between real humans and LLMs. LLMs might not be human, but humans do interact with them as if they were.

*It doesn't matter, of course, whether the accusation is false. Due process means people have certain rights regardless of culpability. But I don't want readers to fixate on culpability, so in this scenario the accusation is unfounded -- let's say someone did break into a house and steal the family jewels, but a security camera identified the wrong person.

No point even considering it if it can't run for extended periods. And actually keeping learning during the duration. As surely person should stay alive for decades if allowed.

In the Republic of Ireland, an artificially wealthy, highly educated, treaty-enmeshed constitutional democracy, the following statement was enshrined:

"4. Marriage may be contracted in accordance with law by two persons without distinction as to their sex."

It doesn't say Natural Persons. My Limited Company has personality but no distinguishable sex.

Thankfully there's the "in accordance with law" phrase so we can sleep ok.

We humans are sleep-walking into ancient nightmares made real through our own curiosity.

What a time to be alive.

Thankfully, there's Mathematics. I made a blackboard today.

As such, all AIs will now be granted as child of the least lawyered up class of society. Poor people.

FYI: It's an "old" article

Putting barriers in place without exceptions for sentience is a recipe for trouble and slavery in the future.

Historical peoples, as a group, may not have been morally advanced enough (though there have always been enlightened humanists in most populations), but modern philosophy and mass education are advanced enough that it should be possible to avoid future pitfalls by not building in exceptions for sentience.

Animal rights are certainly moving towards recognizing non-human sentience; no need to accidentally codify digital slavery for future sentients (AGI).

We're supposed to deny AI rights because it might cheat.

But humans cheat all the time.

We survived cheating humans, why can't we survive cheating AI?

Cheating is the problem: not who does it.