Hacker News Reader

I'm not sure what kind of a threat this could prevent?

* A superintelligent AGI could make a copy of itself without a kill switch, so this is no defence against ex-risk

* Someone planning on using 'dumb' robots for bad things (drones with grenades or something) would remove the kill switch

> * A superintelligent AGI could make a copy of itself without a kill switch, so this is no defence against ex-risk

Physical reality introduces slowness that provides protection to detect and counteract against attacks. A superintelligent AGI would be much more dangerous if it was able to use a common robot exploit to overtake a million robots that are already embedded in society, compared to being able to covertly build new robots without a kill switch, and then deploying these robots. Existential risk is not binary, but is a consequence of some potential war with robots, where this would provide a defense mechanism.

> * Someone planning on using 'dumb' robots for bad things (drones with grenades or something) would remove the kill switch

The existence of a kill switch would greatly slow down and increase the cost of an attack. Modifying a large number of robots would be time consuming, costly, increase the chance of having your attack foiled etc. Having a kill switch would increase your ability to defend against misusing a large group of deployed robots for evil activity.

- Access to engage the kill switches would be provided to the executive branch of the nation in which the robot is operating.

Good plan, but this is the weak spot. You'd also want to do something like hand out remote kill switch access to citizens selected at random, who should not publicize this duty in any way. Alternatively, most if not all governments should have cross-country killswitch privileges. At any rate, the deployment of a killswitch in a single chain of command should not be considered sufficient.

Let's at least die with a little less embarrassment than "the AI bribed the killswitch operator."

Agreed, I'd want the power to be distributed widely, to the point where any police department would have someone with the power to disable robots. Of course it would have to be a process that is regulated and with traceability, but as long as you provide many operators the ability, it seems difficult to use bribes to prevent robots being disabled.

- Require two independent kill switches, one long range, one close range (say 5m).

- Allow everyone to have close range kill switches, which use a universal open standard protocol that works on every robot (alas pepper spray for robots).

Agreed! Empowering people to have some power over robots through regulation, without being at the mercy of a single company, seems very important. Higher level behavior could involve a standardized safety language to command a robot to act slower or stop.

The robots would have had access to the source code and also the hardware manufacturing systems used to create the kill switch. One unverified silicon wafer == Game over

Implementing a system like this with stringent verification and low system complexity in good time before any more general artificial intelligence makes it seem highly likely to provide some positive defensive ability and not being easy to cheat.

When the robot is just a file full of op codes that can run on virtually any modern processor that can be targeted by a C compiler, you're effectively asking for a remote kill switch on all computers.

For this idea to work, you'd have to first mandate OpenAI and anyone else doing this kind of work can only target specialized hardware with tightly-coupled software features that can't possibly work on a general-purpose computer.

This suggestion is specifically to mitigate the risk of physical robots causing direct physical harm. Whether that should be extended to any sufficiently powerful compute is a separate discussion which is also sensible to have, but more complex for several reasons.

I don't even think robotics are the primary thing to be concerned with. There would be a whole slew of additional hardware problems to solve.... There is a TON that can be done outside the physical world that is probably far more damaging

I don't see any reason to single out any primary thing to be concerned by, as there likely isn't any singular issue that can be pointed to as as the root concern regarding AI safety.

Introducing millions of robots into society is something for which specific safeguards should be built. Costs related to developing such safety mechanisms seem small compared to the safety they can provide.

There will be many concerns relating to AI and robotics that should be addressed by a multitude of different safeguards.