Why would that be controversial? It seems to make extremely good sense and even though it may be doubted in some circles I think that most people involved in AI research are painfully aware of our limited understanding of our own psyche.
Neuroscience does concern itself a great deal with low level biological mechanisms and architectures, and is more amenable to cross-pollination with machine learning.
Though I would like to point out that thus far deep learning has taken few ideas from neuroscience.
The reason why psychology is 'too high-level' is exactly what is meant with 'we don't understand it', we're approaching the psyche at the macro level of observable traits, but there is a very large gap between the 'wiring' and the 'traits', some of that gap belongs to neuroscience but quite possibly the larger parts belongs to psychology. The two will meet somewhere in the middle.
A similar thing happens in biology with dna and genetics on the one side and embryology on the other.
The neural networks that were taken 'explicitly from neuroscience' have gone through a vast transformation and that + a whole lot of work on training and other stuff besides is what powers the current crop of AI software. All the way to computers that learn about games, that label images and that drive cars with impressive accuracy to date.
The problem is - and I think that was what the original question was about - that neuroscience is rather very low level. We need something at the intermediate level, a 'useful building block' approach if you will, something that is not quite a fully formed intelligence but also not so basic as plumbing and wiring.
I never said there weren't any developments in neural nets, I'm just saying that few ideas have been taken from neuroscience (there certainly have been some ideas, like convolutions). In fact most things (including most tricks in convolutional neural nets in their current state) a neural net does, we know a brain does not do.
Right now, we have a state of AI that is very much limited by what we know about how our own machinery works. Better understanding of that machinery (just like better understanding of things like aerodynamics, rolling resistance and explosions led to better transportation) will help us in some way or other. And it may take another 50 years before we hit that next milestone, but the current progress all leads more or less directly back to one poor analogy with biological systems. Quite probably there are more waiting in the wings, whether through literal re-implementation or merely as inspiration it doesn't really matter.
To put it in other words: Birds are limited by evolution. They are not an optimal design - they are a successful reproductive design in a wide ecosystem where flying is a tool.
Our intelligence is no different.
This is something Feynman addressed in this beautiful talk (in the Q&A iirc): https://www.youtube.com/watch?v=EKWGGDXe5MA
I think this is one of the key points of the problem: we, as engineers, are expecting this problem (imitating the human psyche/getting to true AI) to be able to be defined rigorously. What if it can't?
I'm not a religious person and I don't generally believe in something inside us that it's "undefinable", but looking at our cultural and intellectual history of about 2,500 years I can see that there are lots of things that we we haven't been able to define rigorously, but which are quintessential to human psyche: poetry, telling jokes, word puns, the sentiment of nostalgia and I could go on and on.
So I would expect strong AI from CS, possibly with a paradigm shift caused by an advance in neuroscience.
1) Acting like a human (this is the Turing test approach)
2) Thinking like a human (this is cognitive science, and for now is focused on figuring out how humans think)
3) Thinking rationally, ie, following formal logic (the difficulty here is encoding all the information in the world as formal logic)
4) Acting rationally. That is, entities that react rationally to their goals (this one is notable because it allows fairly stupid entities).
These are all explained in more detail in Artificial Intelligence, a Modern Approach by Russell and Norvig
Another line for acting human would be the ability to self direct learning in a variety of situations in which a reasonably intelligent human can learn. That means a single algorithmic framework that can learn go, navigate a maze, solve Sudoku, carry on a conversation and decide which of those things to do at any given time. The key is that the go playing skill would need to be acquired without explicitly programming for go.
I believe a lot of our intelligence is the ability to perform solved AI problems given the situation. The key is combining those skills (whether as a single algorithm or a variety of algorithms with an arbiter) and the ability to intelligently direct focus. That's why most researchers aren't confusing alphago with general intelligence. It can play go - period.
Their attempt at a definition that synthesizes all the others is:
> Intelligence measures an agent's ability to achieve goals in a wide range of environments. - S. Legg and M. Hutter