Hacker News Reader

Comment by raindeer3 | Hacker News Reader

How is your concern with the lack of symmetric backwards connections related to the last paragraph about the brain not forgetting? The backward pass is used to both strengthen and weaken weights so in bp the forgetting and learning happens at the same time.

View on HN

Hebbian plasticity in biological NN assumes strengthening happens during the forward pass (the only pass). It is a local phenomena at individual synapses that detect a coincidence (2 inputs occur simultaneously or nearly) and is mediated by calcium influx. NMDA receptors will pass calcium but only if two things happen: (1) they are currently bound to glutamate neurotransmitter (input from their own upstream axon), and (2) magnesium is not blocking their calcium channel (Mg will be ejected briefly from the Ca channel if the neuron is depolarized - meaning currently receiving input from elsewhere). If you are receiving sensory input from both my voice and my face, some set of neurons are detecting this coincidence and strengthening that connection so the next time you hear my voice, it becomes easier to picture my face from all the different faces you have seen.

Here is a good article explaining not just in theory how LTP (long term potentiation) works in neurons but why a particular protocol always works irl (I can attest to the validity of these statements based on first hand experience):

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3843869/

To summarize, the most reliable way to induce LTP is to "take control of the postsynaptic membrane with intracellular Cs+ to block K+ channels, which allows the experimenter to hold the cell at a constant membrane potential and induce a minimal ‘pairing’ protocol to induce LTP: depolarizing the cell to 0 mV while stimulating synapses."

Holding the cell at 0 mV ensures Mg is always ejected, so any upstream stimulation will always be seen as a coincidence.

Yes, but also in hebbian learning you must have some weakening of weights, otherwise the weights would just grow indefinitely? One example I guess is Oja's rule. The difference to bp is just how to select what weights to strengthen and what to weaken based on what information. Forgetting and learning must always happen one way or the other. Or am I not getting your point?

I agree. Above I mention that neurons have finite resources so synaptic strength is essentially zero-sum. When a new set of synapses becomes strengthened, it implies that all the other synapses must be weakened by some amount.

Here is a nice animation of signal propagation in biological neural nets:

https://youtu.be/WCqNn9PEELw

To simulate the dynamics of synaptic strength I created 3D mesh of a dendrite segment with several synapses...

https://youtu.be/tDKUU0SqbSA

Then I simulate the diffusion of AMPA receptors on the surface (the number of AMPAR in a synapse is proportional to its strength)...

https://youtu.be/6ZNnBGgea0Y

I don't have animations of this process but you can imagine what happens when one synapse holds onto receptors longer than the others (has a reduced particle diffusion rate), when there are a finite number of receptors

Maybe Glial cells are playing a role - https://www.sciencedaily.com/releases/2018/06/180618222427.h...

> In cell culture studies, they added neurons to astrocytes that overexpressed ephrin-B1 and were able to see synapse removal, with the astrocytes "eating up" the synapses... "We think that astrocytes expressing too much of ephrin-B1 can attack neurons and remove synapses"

Yeah that would be a problem. Maybe Glial cells play a role, maybe they don't, opinions vary. https://streamable.com/1vi7r

Reply on news.ycombinator.com