Singularity Is Near Full Documentary Michio Kaku – Ray Kurzweil
youtube.com
youtube.com
The thing that makes what these different people are talking about seem so absurd is they pose theoretical situations, and ask "What If?", and then just stop right there and don't critically examine the situation at all. All the "What If?"s then drive a sort of nagging horror at all the possible negative outcomes, and then you become Ray Kurzweil.
The biggest fallacy of the Singularity is the accelerating change hypothesis. In order for accelerating change to work, you not only have to invent new things, you have to adopt & apply them. Recently there was a story posted about how it took hundreds of years after the discovery of stainless steel for it to begin to be adopted by companies and individuals. And the reasons for the lack of adoption are almost always due to one thing: limitation of the human brain.
We still don't know much about the brain. As long as everyone's brain remains stuck in this meatsack and evolves at a very slow rate, so will the social adaptation of technological change and the investment therein. And that will limit not only our ability to develop AI, but how and where we use it.
To me it's far more likely that a true AI will first be used to manufacture cheerios than control a military weapons system, even though the military weapons system is where the biggest dollar investment in AI will be. The cheerios manufacturing plant will have a practical problem that only AI can solve, meanwhile the weapons system will probably only use AI for a single task and it'll be phased out in favor of a less buggy, cheaper, more effective weapons system.
With regard to human intelligence, Augmented Intelligence has already extended human minds beyond the confines of squishy flesh bags.