This will invalidate fantastical accelerationist woo until such evidence is presented, and -in the current state of the art research within relevant fields- it's not even certain we can know what would constitute such evidence _in principle_.
This will invalidate fantastical accelerationist woo until such evidence is presented, and -in the current state of the art research within relevant fields- it's not even certain we can know what would constitute such evidence _in principle_.
If the expectation of a superhuman AI takeover is built on five key assumptions that have no basis in evidence, then this idea is more akin to a religious belief — a myth
My problem is, even this is wrong. I do not find the "five key assumptions" he cites to be the foundation of beliefs that a dangerous "Superhuman AI" could emerge. So to me, this article simply fails right from the jump. The thing he is arguing for and/or against just don't, collectively, constitute a well-developed argument for the overall thesis. Not in my opinion anyway.
To go through them one by one:
Artificial intelligence is already getting smarter than us, at an exponential rate.
I don't know anybody making this claim, nor is it apparent that belief in this is necessary to think that dangerous "Superhuman AI" could be a problem in the future. I guess it comes down to what you think Kelly meant by "soon" in the preceding sentence.
We’ll make AIs into a general purpose intelligence, like our own.
It's the "like our own" part of this that I have a problem with. Yes, I think many people believe that we will create some manner of "general intelligence", but I also believe that a lot of researchers acknowledge that it may not be 100% "human like". But this may be my own bias showing, as I personally harp a lot on the importance of the distinction between "human level" intelligence and "human like" intelligence.
We can make human intelligence in silicon.
This is probably the most sound one of all, but here's the thing... saying "there is no evidence for this" is simply wrong. To hold to this is to completely ignore all progress made building AI in silicon to date. And while we don't yet have "general intelligence", I don't see how one can just ignore incremental progress completely and say "there's nothing there". I would argue that the null hypothesis should probably be that general intelligence on silicon is possible that that the burden of proof is on the people arguing the opposite.
Also, not everybody thinking about / working on AI is focused on the "in silicon" approach. The majority? Yes. But I'm sure there are a handful of people working on ideas related to DNA computing, chemical computing, growing synthetic brain matter, etc.
Intelligence can be expanded without limit.
To me, this may or not not be true, but it's completely irrelevant to the thesis at hand. The question isn't "can machine intelligence be unbounded?", but rather "can machine intelligence exceed human intelligence?"
Once we have exploding superintelligence it can solve most of our problems.
Again, this strikes me as mostly irrelevant to the question at hand.
So yes, this article raises some interesting points, but I don't see it accomplishing what it seems to purport to do.
It could be that there's no way to do that in silica without outstripping plausible bounds on hardware size or power use, I don't know...
I sense that your arguments are in concordance with TFA, broadly speaking. Also, I suspect you read it rather uncharitably for reasons having to do with tone or stated thesis. Either that, or my reading comprehension is failing me. Maybe I want it to be more convincing that it really is, as i already laid bare my sentiments towards accelerationist woo.
I'll take that as a compliment!
All joking aside, I'm not saying every word of the article is wrong or anything. Maybe I'm just being too literal or pedantic or something. It just seems to me that the article sets out to make one very specific claim, and I just don't think it succeeds in making a strong case for that specific claim. Is there some interesting discussion there? Yes. Are there things that Kelly says that I agree with? Yes, absolutely.
So maybe we're not too far apart on this after all.