Should we strive to simplify neural models? | Hacker News Reader

Should we strive to simplify neural models?

nature.com

47 pointsnefitty6 comments

Having a physically realistic model could at least serve as a point of reference to see how various abstractions and simplifications differ from that baseline.

There are no revelations and everything worked out as expected, so this experiment is a failure? How? And now you need more money for funding studies, because reasons such as it's too hard?

I'd hope these scientists would delight in nuance, but it seems as the pursuit for a major breakthrough has ruined their own brains. Build the right thing, build the thing right. And please stop bull-shitting. If nothing happened, that's fine. If you need ten more iterations of Moore's law, then say so.

But don't fake-it-until-you-make-it... pretty sure that only leads to bad science.

> For example, a single cortical neuron receives input from thousands of other cells, but it is unclear how it processes this information.

Is this true? If so it sounds like one of the most pressing research questions in AI, or perhaps even all of science...

Yes, it's true, for certain definitions of "unclear" and "processes". The whole issue here is that the mind/brain is too complex to describe at any single level of abstraction. There are just so many different, valid levels of description, all the way from physics to high level computational approaches. Wikipedia lists 22 distinct branches of neuroscience [1], and each has their own set of problems they are trying to answer.

The problem this article brings up is that one researcher (who has received a lot of money to fund his project) has come up with an explanation of "process" that is "clear" to him for the questions he is interested in. However, for everyone else, "process" means something different, and they are unable to see how to translate his model to answers questions they are interested in. Without a simplified model, one that abstracts away the huge complexity, there's no ability to generalize its conclusions beyond the exact simulation at hand, and this means it has virtually no explanatory power.

So yes, because there are many valid definitions of "process", that statement is true for many of the interesting definitions.

[1] https://en.wikipedia.org/wiki/Neuroscience#Major_branches

> the mind/brain is too complex to describe at any single level of abstraction

Sure, but now we're talking about a single neuron...

> Wikipedia lists 22 distinct branches of neuroscience [1], and each has their own set of problems they are trying to answer.

Sure, but only one or two of those 22 are about single neurons...

In artificial neural networks neurons are approximated basically as a weighted summation and a thresholding/sigmoid/rectified linear operation. It would be interesting to know how far from the truth this is. Is the biological neuron doing something entirely different than what our artificial neurons are? That's what I feel would be a pressing research question.

> Sure, but only one or two of those 22 are about single neurons...

I count four or five, but I agree that may not be the best example for my point, which is that (even within those few subfields) there are many valid levels of description.

> Is the biological neuron doing something entirely different than what our artificial neurons are?

Yes, very; they are highly non-linear and have complex timing dependencies. This illustrates the point. Once you model all those intricate biological details it's unclear how to translate that into a mathematical function that we can understand for the purposes of higher level computation. Not to mention, we're still talking about spiking models, so we'd also have to abstract away spiking in order to look like an artificial neuron. The gulf between the level of description that you and I want and what biologically realistic models provide is huge.

Reply on news.ycombinator.com