> Superhuman machine intelligence is prima facie ridiculous.
> - Many otherwise smart people, 2015
I was among those criticizing Sam's point of view, but not because I think the idea of superhuman machine intelligence is ridiculous. I don't know one way or the other whether we'll achieve it or not, but I don't consider it impossible.
What I, and I believe many others as well, was criticizing was Sam's insistence that the current techniques have the potential to cause such great harm that they need to be regulated and that extreme measures need to be taken to address the possible harm of modern AI and ML techniques.
This type of fear-driven reasoning is wildly out of proportion with the facts, as anyone who does research in the area of is familiar with the research in the area can attest to. Modern machine learning techniques are heuristic methods to find optima in functions over high dimensional spaces. Neural networks and support vector machines produce impressive results on image classification and other well defined problems, but are far from general purpose techniques that can be easily applied to any given problem.
So, the bullet points:
1. Current ML techniques aren't anywhere near creating general intelligence, nor are there areas of research that appear likely to yield superhuman levels of intelligence in the near to moderate-term future.
2. This doesn't mean superhuman levels of machine intelligence are impossible, just that we don't have any current methods that are likely to lead to them.
3. Taking action against an ill-defined, likely phantasmic threat is neither cost effective nor helpful for the progress of AI and ML.
4. When the people who have real, expert knowledge of something all tell you one thing and the people who have something to gain from getting your attention by promoting sensational opinions and cherry-picked facts tell you something else you should rationally assess their motives and weight that when deciding who's views more closely approximate the truth.