Comment by georgemcbay | Hacker News Reader

Comment by georgemcbay | Hacker News Reader

Parent Full thread

> They're fixated on AI itself as the risk ("alignment!!!1!1!!"), as opposed to what humans with these tools will do.

What really is the difference? Aligned AI wouldn't help humans do these things. But we've failed hard on aligned AI at every level and will continue to fail on it, as far as I can tell. We were likely doomed to fail because of the impossibility of coordination combined with the fact that there's no natural gating factor to slow anyone down.

FWIW I probably mostly see things closer to the way you do than they do, I'm just not sure there is any point in drawing distinctions.

IMO when AI kills us all it is probably not going to be a malignant action, and I don't even think it will be AI assisting humans at efficient killing, I think it is going to be a complete accident. Someone is going to trust the god machine too much due to an advanced form of the eliza effect and hook it up with direct control of a system that can do real damage and an epic oopsie will occur at a speed beyond which a human can stop it. Not because the AI wants to kill us, but because it is incapable of the empathy required to care if it does, combined with our ironic inability to not anthropomorphize it.

But I guess ultimately the exact reason isn't going to matter much.

You don't need mis-aligned AI to program a drone to fly to a specific address, loiter, and then dive bomb the first person whose face matches a predetermined photo. This was doable in principle with tech from four years ago. What's changed since four years ago is that you can run the facial recognition and visual navigation algorithms in a cheap onboard chip on a drone.

EDIT: Russia is already doing this, per an NYT article from a few days ago, though they're targeting infrastructure (find the first kerosene tank and fly into it) rather than specific people.

in your hypothetical 300 million person scenario why even bother with individual face recognition?

If you're going to kill everyone you don't need a system to discriminate individual targets, you just need to recognize any target broadly which you can do with even older technology.

Yeah, if you're actually trying to commit genocide, yeah, don't bother with facial recognition. But in a first strike you'd want to make sure that you actually kill the entire military command and control apparatus, and just to be thorough you'd probably want to individually target, say, every single officer and senior NCO in the military. Can't have a general escaping because two drones both targeted his driver by accident.

This is an incredibly dumb scenario that belongs in a cheap sci-fi novel because it hand waves away human motivations (Who should do this and to what end? What is there to achieve with something like this that cannot be achieved some other more conventional way?) and feasibility both in terms of actually getting the hardware ready and completely ignoring counter measures.

Which is to say: there are significant downsides to such an approach to the extent that this is fit for a spooky story or an abysmal horror flick but not compatible with actual reality.

Cheap drones do change a lot about how wars are fought, sure. They are in that sense extremely relevant. But that does not mean that your scenario or similar scenarios like it are remotely realistic.

Put another way: Cheap drones are not nuclear weapons (which actually did lead to a earth shattering change in global power dynamics, though I would argue even their transformative hasn’t been absolute).

Cheap drones are more like machine guns, tanks or airplanes. “Mere” new technology. Obviously hugely important and impactful, not earth shattering.

(As for whether AI might be earth shattering, I do not know nor do I even want to get into that question here.)

Human shaped things that emit cellphone signals is a cheap and effective target profile for this kind of attack.

Reply on news.ycombinator.com