As an aside, assuming a doomsday scenario, how long can secrets like this stay outside of the hands of bad actors? On a scale of 1 to enriched uranium
This will be solved like all other engineering and science: with experiments and iteration, in a controlled setting where potential accidents will have small consequences.
An unaligned system isn’t even useful, let alone safe. If it turns out that unaligned AGI is very hard, we will obviously not deploy it into the world at scale. It’s bad for the bottom line to be dead.
But there’s truly no way out but forward; game theory constrains paranoid actors more than the reckless. A good balance must be found, and we’re pretty close to it.
None of the «lesswrong» doomsday hypotheses have much evidence for them, if that changes then we will reassess.
I have no overall position, but climate change and nuclear weapons seem two quite strong counterexamples to this being a sufficient condition for safety.
Also non violent.
I think if we have a major AI induced calamity... Then I worry much more. Although... Enough scary capability in a short enough period of time... I could see violence being on the table for the more radical amongst the group.
Your concern is very interesting though, and I think important to consider. I wonder if the FBI agrees.
The AI doom cult is not a peaceful movement. When an extremist tells you what they will do, you damn well listen to them.
> The thing that most worries me about technology is not the technology itself but the greed of those who run it.
Someone slimy with limitless ambition like Altman seems to be the worst person to be in charge of things like this.
You see it from the all commercial deals, protecting company image more that eliminating threats, or even from this post, if it is true.
Not telling about breaktrough in research if it would end the deal with Microsoft.
Which threats exactly did he ignore?