History. We have had a period of adaptation and experimentation with every other invention in history. This one likely will be no different.
> Won't it just be used to augment the amount of harm that humans can do?
Sure, but that's everything that makes us more capable. It also makes us more able to defend and create.
Right, I learned from Napoleon about the dangers of AI...
The large land mammals were doing pretty well too, until a bipedal superintelligence showed up.
History is a get thing to learn from and a very poor thing to assume is going to repeat.
The worry with superintelligent AI is that it'll dethrone us, and automatically make it the most dangerous thing on the planet. In particular, more dangerous to us than we already are to each other.
But as is, all you're doing is making theories, and there's not much of any precedent for any of them.
lots of things ramp up without our notice. Like CO2 in the atmosphere, or Ozone depleting chemicals, or PFAS in the water, microplastics in the ocean. I don't think it's a small list.
They did, to some extent, see it coming (in observing a threat becoming more prevalent and dangerous). But they did not have a way to stop humans. Meanwhile, I'm just staring at the power switch on my computer, wondering how we'd stop AI.
(This is not an argument for unrestrained AI - we should safeguard, absolutely. It is a counterpoint to your "we won't see it coming" argument)
This is we talk about Moloch commonly on HN. We see systems that are going to have obvious bad outcomes for most individuals, and be unaligned with what most humans want, and yet the system persists and even grows worse. This is how I believe it will be with AI too. It will make the rich insanely rich and powerful even at the risk of destroying us all.