I don't get this scare about AI. Imagine, you are a genius with IQ 200. You have studied hard your entire life and you are now an expert in several fields. One day you decide to conquer the world. Will you succeed or not?
Assuming you have 200 IQ, if you could make thousands or millions of clones of yourself quite cheaply, you still might not succeed in taking over the world, but it is no longer a laughable idea.
Overall, I’m somewhat ambivalent about the possibility of x-risk, especially if we work reasonably hard to prevent it.
But it shouldn’t be ignored. AI is moving very quickly and it is unclear how powerful its capabilities will be in the next 10-20 years. Of course, there are many other risks presented by AI that we need to stay on top of as well.
Copying AIs just results in effectively higher intelligence (bigger brain). Things would be different with self-replicating robots of course, but duplicating robots is not that fast and one could argue that the robots are dangerous rather than the AI controlling them, because they would be dangerous even if controlled by stupid non-AI program.
Earning money, strategizing, researching, smooth-talking humans, playing politics, and so on. Potentially even improving itself.
These activities could give it power and influence.
Name the next most intelligent animal to humans...
Have humans completely and utterly conquered them?
The intelligence explosion already happened and humans were it. Now you're questioning if it's possible with intelligence explosion 2.0?
How about 500 iq with a direct line into the entire information retrieval and processing system of 7 billion humans, most of whom are already easily manipulated by relatively rudimentary social media ML.
It's not about 200 IQ, it's about when we reach bigger gaps than that. Though yes, I suspect if the 200 IQ guy can copy himself at will, get hardware to do years of 200IQ thinking in an hour, etc. he might succeed.
The hairless monkeys were embodied and violent and they still took millions of years to dominate. The question is whether intelligence on its own provides sufficient advantage to be immediately dangerous. How can you conquer the world by doing "years of 200IQ thinking in an hour"?
I'm pretty neutral about it at the moment, but it's not inconceivable that something with an IQ of 400 and can think 30x faster than you, has no need to sleep and has 500x the working memory of you would be dangerous if it didn't like you.
In fact, your comment and others like it make me wonder if people are just in denial because as I said, it doesn't seem impossible?
I wish I were kidding.
This view is just pure human ego talking.
Dolphins probably think we're fucking morons.