It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
It's a little contrived but you tell it 'solve world hunger', so it 'does a Thanos' and wipes out half the human population by releasing a pathogen or something, so it's fulfilled it's primary function but (hopefully) not in the way you expected.
Then there's the problem that humans hurt other humans. Should the AI stop that? It's going to have to hurt humans to do it. But if it doesn't, that will hurt other humans...
Isaac Asimov made a career out of pointing out the hubris of it. Three simple laws, what can go wrong?
I'm not even close to being an AI alarmist, and I'm skeptical of a lot of Nick Bostrom's arguments. But he does do a pretty good job of articulating the problem with this scenario in his book Superintelligence. He makes a good case that it would be very difficult to articulate such values for the AI. If you're interested in this topic in the general sense, I'd suggest reading the book. I don't think it's perfect, but I will acknowledge that he makes some good points.