Hacker News Reader

Comment by minihat | Hacker News Reader

This is exactly right.

I started a PhD in 2017 studying neuroscience + ML. I thought studying the brain would help me understand ANNs better. I was wrong. Ended up applying ML to analyzing EEG, MRI and similar.

View on HN

Is this because we're misapplying the analogy to ML? I.e., in an effort to communicate and understand ANNs, we "pretend" it's like a brain. Just like before, we used "file retrieval systems" to understand the brain, or electricity is like "water in a pipe", which are also wrong. Analogies often only go so far, beyond which they do more harm than good.

What you're describing is endemic across HN (and tech, tbh). Lots of people on here "know" computers/programming/CS very well. They, naturally, tend to use analogies to computers/programming/CS when trying to explain or "think out loud" in their comments. That's fine. It's what they know. The common problem arises when people forget they're analogizing and begin to see their analogy as ontologically and conceptually identical to the thing they were making an analogy for. This requires a certain amount of ego, echo chambers, and self-valorization, so that they never have to face the actual issues with these analogies.

But as many comments here have pointed out, studying neuroscience, for example, usually makes those analogies seem painfully inadequate. The same is true in philosophy of mind, for example.

I'm sure that there exist people who get lost in the analogies. Practitioners are generally not confused that ANNs are simplifications of the brain. The questions are which simplifications are most relevant and whether complexities can be added that yield better results. My own research was about reintroducing absolute location. I'm standard ANNs location is relative within a graph model of the network. I'm the real brain blood vessels and other macrostructures deliver materials used to grow and modify the neurons and these affect the network based on physical location. I'm fact, by adding these back in we bypassed the XOR limitation (i.e. Minsky's result leading to back propagation). Concretely, we observed learning XOR over the inputs within a Hopfield network using Hebbian learning modulated by spatially modulated trophic factor).

Turns out "water in a pipe" is actually surprisingly accurate, just that the "waves" are sloshing around at half the speed of light.

https://youtu.be/2AXv49dDQJw?si=ZxRyiu8WwynUXwf_&t=655

I believe, at its best, it’s an incomplete model (which may be enough for most people). But it leaves out important aspects like magnetic field work and probably a host of aspects from quantum theory.

https://www.slideshare.net/slideshow/georgia-state-universit...

Ok, yeah, I'll agree with incomplete. Only useful for considering the bulk flow of electrons assuming no bulk EM field.

Thanks for the read.

Have we hit the limit of the analogy, or have we hit a limit in our understanding? Both neural networks and actual brains have behaviors that emerge from the interactions of smaller components. Neural networks have trivial connections compared to brains, but our understanding of the emergent behaviors seems very limited. To me, this is a sign not that the analogy has reached a point of breaking down, but that our tools aren't sufficient to work on even then trivial connections. I do expect the analogy will break at some point, but I'm not sure we have reached that point yet.

I hoped neuroscience, as a field, was on the cusp of a physical theory of learning and memory. I dreamt of an intersection of information theory, neuroscience, and ML.

Alas, state of the art in neuroscience / neural engineering is closer to bloodletting than a mechanistic theory of learning and memory.

Good thing there's not a lot of ML being done in Haskell... imagine a Brain Monad tutorial.

Reply on news.ycombinator.com