I think AI - like all changes - will lead to some bad things. The internet did too!
But I don't think AI will kill us all.
Interestingly I'd note that the two outcomes you listed (power concentration and loss of control) are dimensionally opposites!
For me this just shows that the future contains such a vast array of possible outcomes that focus on the negatives completely missed the positive outcomes that future also holds.
The industrial revolution produced economic disruption.
More recently I remember there were typing pools at my dad's work and there was serious concerns about what jobs women would do because personal computers meant they weren't needed.
Turns out in both cases people were blinded by biases around future work looking like current work.
It is good to maintain some optimism that people will figure out what to do but there is a difference from rushing towards such a future and blindly assuming it will be good by default. Are there things you think we need to figure out to avoid a bad outcome?
> Interestingly I'd note that the two outcomes you listed (power concentration and loss of control) are dimensionally opposites!
One can imagine a world where both happen. For example, an arms race between two leading labs causes them to accumulate a majority of the world's compute and market share for intelligence, with the race dynamics at some point resulting in a catastrophic incident of a self-replicating botnet taking over a large part of the internet due to a rushed, reckless Hugging-Face style RL run.
We focus on stopping bad things because people and systems that don't prevent bad things tend to stop existing. A million good things can happen yet be rendered permanently in vain if one bad terrible thing occurs.
I think we should stop the bad things.
We don't stop building cars because there are crashes, we build better safety systems.
I'm against the doomer narrative ("AI will kill us all") not against a clear eyed approach to making safe systems.