Hacker News Reader

Comment by incadenza | Hacker News Reader

I generally agree. I think moral consequences become especially relevant in the field of AGI.

Computerphile has a series of videos on AGI that touches on this topic, using the lens of "define a human to a computer," a seemingly simple task with a huge slew of ramifications that come with it. Basically exploring whether or not Asimov's Laws are actually useful or practical (they aren't, and this is a huge reason why).

The short version of the specific video in the series about this explores how edge cases become extremely relevant in this domain, and asks whether or not an AGI would consider these to be people or sufficiently people-like:

* Dead people

* People in vegetative states

* Unborn people

* Simulated human brains

* Sufficiently intelligent animals

And so on. We can't simply list every edge case, that list would be literally endless and the developer is guaranteed to miss one that the AGI will eventually find.

They all come with implications. How can an AGI never harm a human without also accurately predicting extremely discrete details from the future? Any action it takes may cause harm or death to one or two humans 50 years later, and it can't really know for certain. It can probably known if its actions will affect hundreds or thousands, but not one or two.

Suddenly a whole lot of deep moral judgements have to be made, some of them nearly impossible for a human (e.g. the aforementioned unborn human example above). The Three Laws of Robotics unfortunately rely way too much on human intuition, something that is extremely difficult to model in a machine.

I may be remembering incorrectly, but the whole point of that story was how those laws fail spectacularly. That's a critical detail. But I digress.

Clearly an AI designer isn't qualified to just make those judgements on their own, and probably doesn't want to anyway. Really no one, either living, dead, or unborn, is qualified to be the judge and jury in this case, and a committee isn't necessarily better.

It's a fascinating topic, to be sure, but I'll admit that it's not my area of expertise.

Agreed, it's a fascinating topic. I've been reading Bostrom's Super Intellegence which has been a great introduction to this range of problems.

Asimov's Three Laws of Robotics are often remembered as being hard set rules, but Asimov more often than not used them as the precursors to murder mysteries. So many of the best of his Three Laws stories are locked room murder mysteries where a robot finds some edge case that no one had yet considered. Especially, the entire concept of the Zeroth law is interesting because it was a loophole defined by robots, and the ethics of which are somewhat debated in the series, but most of that debate is left as an exercise to the reader so readers don't often do it. But even a lot of the very edge cases listed here are part of the meta-ethics debates across the Three Laws stories.

Whether or not the Three Laws are a viable system for AGI governance (and I agree it seems rather unlikely, at least in their simplest define forms most encountered in Asimov's stories, but even the stories would claim that the way they are written in English is not exactly how they are coded in positronic technology, however that may be), the ethics conflicts and edge case debates in the stories are very fascinating reads.

It is somewhat surprising to me how many people seem to think the laws are set in stone and don't realize how many of the stories themselves are secretly ethics debates about edge cases. Just because some of the characters believe the laws to be infallible doesn't mean those characters are right and the laws are infallible. There wouldn't be so many tales about the Three Laws if there weren't so many edge cases and exciting locked room mysteries to explore.

Reply on news.ycombinator.com