It's like trying to build an airplane by studying hang-gliders instead of aerodynamics.
AI researchers spent decades trying to formulate definitions and models purely theoretically, and it turned out those theories were not super useful in building out modeling real systems. The toy models are essential.
I suspect someone will get some level of mega-expert system/simulated intelligence going sooner than later. Maybe something that can handle the types of commands you could give a dog and it being able to use enough fuzzy logic to work basic things out like, "Get me a coke from the fridge and then get my slippers."
That seems a lot more likely than suddenly birthing human-level AI from NN's. I think NN's will ultimately fail the same way planes work nothing like birds. Trying to copy a biological system very closely just doesn't make sense, at least most of the time.
NNs will almost certainly succeed where expert systems failed, largely because we now understand that no single monolithic pattern matcher will suffice alone. Any cognitive engine must be composed of many components, each attuned to a different purpose and context. And now we better recognize the huge need for learning, both for initial skill acquisition and for lifelong thereafter.
Minsky's "Society of Mind" is probably a better illustration of how AI will evolve (if not manifest), as well as how it must integrate with our myriad collective needs and personal lives.
The success of fixed wing aircraft is not an ideal metaphor for the failures of AI research. Nor do the successes of neural nets herald the biomorphic approach, since they don't really resemble the brain very much.
Thus, I fail to see why some sort of fundamental understanding of intelligence is required in order to create it. On the contrary, it's not hard to imagine genetic algorithms combined with neural networks being used in a similar fashion, provided there's sufficient data and computing power available.
Are attributing that position to the author? I don't think that's what he's saying at all (italics mine):
> The frontier machine intelligence architecture [at] the moment uses deep neural nets.... Silicon brains of this kind...have recently surpassed human performance on a number of narrowly defined tasks...We are learning how to tune deep neural nets using large samples of training data, but the resulting structures are mysterious to us. The theoretical basis for this work is still primitive, and it remains largely an empirical black art.
Doesn't this go back to the second paragraph of the article?