Nicholas Carr’s 'Glass Cage': Automation Will Hurt Society in Long Run | Hacker News Reader

Nicholas Carr’s 'Glass Cage': Automation Will Hurt Society in Long Run

pbs.org

26 pointsrazorburn30 comments

A friend's Roomba committed suicide. He got home one day, and the apartment was filled with steam. Walked into the bathroom, shower was on, with the Roomba just sitting there getting soaked. It had managed to pull the hose while hoovering, and the hose yanked the valve handle.

My friends' Roomba often seemed like it was trying to kill itself. I've had to untangle it from its perils far too often to think its an accident. Nothing says "maybe we should move to the country" more than a robot trying to persistently kill itself with whatever macbook cables it can find, ingest and use to flip itself over, grinding over and over on the insulation until it finally gets that .. one .. last spark .. it needs to end it all. Seriously, happens often.

The moral decision is made by the person who released on their floor a semi-autonomous machine that cannot make moral choices. Calling this "robot morality" is idiotic, akin to saying that because an ax will indifferently cut a log or a human neck, it decides that cutting one is morally equivalent to the other.

That was actually the article's point - by releasing this robot of death, you cede moral authority to it. Or by your reckoning, you make the decision to kill; the robot just disguises that fact.

But it's sensationally worded. That's true any time we take an action that may have unintended consequences, and what that action is — whether it's releasing a robot vacuum cleaner, striking a log with an axe, or writing a book with potentially dangerous ideas — is irrelevant. My point is that smart people say dumb things when recent technology is involved.

The idea is, we've not had robots to do our killing for us in the past. This is something new. You can kill (bugs in this case) without being present or even knowing its happening. Worth an article I think.

  The idea is, we've not had robots to do our
  killing for us in the past.

A bit of trivia I picked up in a robotics class: Most of the common definitions of 'robot' include cruise missiles, which we've had since the 1970s at least.

Setting out poison, traps, or mines, as we've done for centuries, is pretty similar; it's meant to kill in our absence, and can easily go wrong.

I think, though, that you're getting caught up in quibbling over choice of words and, in the process, missing the point of the article. It's not really advancing the conversation, it's just pedantry for the sake of pedantry.

There is an interesting difference, though it's not the one you've chosen to frame your comments around. It doesn't matter that the Roomba has no more moral agency than a sack of hammers. That's a point the author passes by on the way to the real crux of the article, which is that autonomous machines have a way of masking this fact in a way that tricks humans into ceding their own moral agency. That example of a person who wouldn't knowingly vacuum over a cricket but happily uses a Roomba is just a less politically-charged proxy for the question of autonomous military robots carrying weapons.

The point is that "unintended consequences" are much more likely with complex systems. When you wield an axe, the consequences are fairly direct. When you leave home for the 101st time after buying a Roomba, you are still responsible for the consequences, but they are much harder to keep track of.

Sensationally worded?!? You're saying there's a chance the headline "When Roombas Kill" could have even a single trait in common with linkbait? Outrageous!

This is a valid point, but there is another interpretation that once technology advances to the point that its pure magic to the general public, the general public can no longer have an intelligent discussion about the morality of their vacuum cleaner, they have to go all mythological, which occasionally is going to look pretty comical to the enlightened non-general public.

Or another way to look at it, is enough technological progress and specialization results inherently in a fragmentation of what morality means because the concept can't fit the same way in everyone's head (square peg round hole). Its a natural result that some fraction of the population will just meaninglessly babble while other fractions laugh. Its possible to interpret this has already begin with respect to evolution, geology/age of earth, climate change, stem cell research, euthanasia, gay marriage, etc...

Seriously, the roomba doesn't make a judgment about the insect because the roomba isn't making judgments about ANYTHING, it's just an extension of a human's choices, it just happens at a delay from when the human initially sets it up.

This is as much a question of morality, as saying that a vending machine that crushes an insect in its inner workings is a question of robot morality.

It's a question of morality in a philosophical sense, not as an application of science or engineering.

The roomba is capable of killing. Whether the operator is aware of this or not, when they use the roomba to clean their floor, they're ceding their moral agency to the device and to its creators.

The logic can be abstracted from the roomba to any other device. This creates the need for device operators, whatever the device, to examine the potential outcomes of device operation and potential assumptions and motivations of the device's creators.

Failure to examine the assumptions and motivations is the same as ceding moral agency, deciding not to decide and throwing hands in the air and saying "well, didn't see that coming" when the creator's assumptions and motivations play out in a way contrary to the operator's morality.

So, yeah, it's about a roomba killing a bug, or a vending machine spilling a soda. It's also about a facial recognition device not noticing a black guy on a black background, or a drone killing a dog because it's barking aggressively.

I'm sorry, I don't see those in any way the same. In both of your latter examples there IS a judgment being made by a form of AI someone built, there's a question there of someone building a tool that can act and decide things on its own. The roomba and the vending machine don't decide anything, they're just extensions of a human making a decision. That's not "robot morality" any more than it's "sword morality" or "hammer morality".

Right, in the case of the roomba and the vending machine, the human decision is 'turn on the roomba' and 'push the vending machine button'

The assumption of the roomba creator is that 'anything on the floor can be vacuumed, living or otherwise.' The moral choice, with regards to the life impacted by the roomba, is to disregard the potential value of the life. It's all debris.

The assumption with the vending machine is similar. The only value considered by the creator is in the transaction involving the operator's money and producing the material the operator wants. Anything else is considered without moral value.

Morality can be as much about the lack of value placed on something as it is about the value placed on something. I'm talking about moral agency, the implications of operating a device, and not about how a robot "feels" about doing something, for the lack of more accurate word.

> We’re going to have to figure out how to give machines moral codes even if it’s not something we want to think about.

The author of the linked article may want to read some Asimov.

"Guns don't kill people, people with guns kill people"

The source interview is much more interesting than this blog's summary: http://www.pbs.org/mediashift/2014/11/nicholas-carr-glass-ca...

Cached: http://webcache.googleusercontent.com/search?q=cache:zLvbJ9R...

The article seems overly sensational, but the point is fairly interesting - people do not think about unintended consequences of delegating decision making to machines.

While the crickets' fate is unfortunate, this article made me think about self-driving cars and all the wildlife that gets run over every day. For the larger animals, say a deer, the autonomous cars seems to promise the same safety as to a pedestrian, but what about smaller animals?

When people swerve to avoid a squirrel or a bunny, they can (and do) cause accidents that result in greater damage. It would be very interesting to know how the autonomous system is programmed to act in such a scenario?

That was one of my early thoughts about automated cars: Will we see wildlife return to some areas in greater numbers? Even a small decrease in say mountain lions in LA killed by cars might cause a large increase in breeding over a few years.

What about areas with a lot of deer? Might they become even more overpopulated? Will more starve? Will they be driven even further into settled areas?

Even small changes in the number of deaths might have notable effects.

So your thoughts are that roadkill is an effective population control for deer? Not really seeing this.

For the mountain lions, that would be overall positive from my view, but of course carries the risk of attacks on pets and humans. These can be mitigated with education, such as don't let your toy dog out unsupervised at night. Lived next to a nature preserve for a couple years and had a mountain lion visit the property on a few occasions. Never had a problem. Rattlesnakes... that's a completely different story...

I looked at some numbers for Michigan. Hunters take ~400,000 deer each year, while there are ~50,000 vehicle-deer accidents (I guess the majority of the collisions kill the deer).

I haven't found a convenient source for predators. There are wolves, and coyotes, bobcats and bears. I guess bobcats, coyotes and bears mostly don't go after adults, but they can have a big impact on the number of fawns.

Not the overall deer population, but around the margins, on the edges of settlements. Seems like it could make a difference there.

That brings to mind a study that I saw summarized, where the researcher put a fake turtle in a road and recorded the reactions of real drivers. While some would swerve to avoid, and a select few stopped their car and got out to help it across the road, there were also some who actively tried to hit the fake turtle with their cars.

I would assume that the automated system would overwhelmingly avoid any obstacle in the road, living or nonliving, except in situations where avoidance would create a greater danger to vehicle or occupants.

A slowly moving animal that does not occupy a large fraction of the lane width, is lower than the under-body clearance on most cars, and does not suddenly change its direction would overwhelmingly prefer automated vehicles. Squirrels, on the other hand, have a tendency to freak out and try to dodge when the car gets closer. The car might adjust to avoid one, only to have it juke underneath a tire at the last second. I doubt the car would do much more than to slightly adjust speed or move laterally within the bounds of the lane it currently occupies.

Great for turtles, but squirrels and rabbits don't have quite the right behavior profile to benefit from the likely collision avoidance strategy.

Let's try a car analogy.

Suppose you're driving a car. If a cricket crosses its path, the cricket gets flattened to its death. But it's not because people place no value on a bug’s life; it's because the cricket is hard to see, and even if it can be seen, there's no time to react.

The same with a Roomba: the person setting up the Roomba doesn't see the insect, and even if it can be seen, there might be no time to react.

How about those poor Roombas out there forced to suffer a life as a Doomba? https://www.youtube.com/watch?v=PSOkNr0X4ZU

Socrates: This writing business 'Will Hurt Society in Long Run'

Roombas don't kill insects, people do.

Reply on news.ycombinator.com