So it is with any technological innovation. Should computers not have been invented because they eliminated jobs? Should steel? What about agriculture? The future is sure to be different, but that doesn't mean we should fight to deny progress. That way lies the Luddite and Conservative. It's only possible to use new tools for good, not try to erase them to prevent evil.

I think that many technologies, ones that we continued using, are good. The ones that turned out to be bad, we banned. If you make a list of technologies we are still using, then it will contain good ones.

I think that actually, we would be better off as humans if we could figure out a way to coordinate (not easy) and decide in advance which technologies we allow to be released into the world. It wouldn't be perfect, as we might still make mistakes, but we maybe could have stopped leaded gasoline, CFCs, Thalidomide, social media feeds, and so on.

If it's a big enough evil, yes, sometimes we should not invent some things. And I say this knowing that there are currently drugs that the FDA is holding back (due to over-caution) even though they would be very likely to save lives. Sometimes we don't take enough risks. Sometimes we take way too much.

I don't know if stopping unaligned AGI is possible, but I think it's worth trying. I can imagine some good coming from aligned AGI, but I feel like most of the things we could do with aligned AGI, we could also do with just regular old narrow AI, but slower. Slow sucks when people are dying, but if there is the possibility of EVERYONE dying and nobody new being born ever again, then that's the thing to avoid.