Decentralized Reinforcement Learning

In the context of neurobiology, the term "type of neuron" has a specific meaning referring to the cell type of the neuron. The brain is host to many many different cell type populations. But I see now that you're not talking about this at all.

My second point:

I think we're running into a semantic/ambuiguity issue here. I'm not sure what you mean by "type" or "kind" and "functions".

For example:

> I'm not aware of any neuron that functions differently from other neurons.

Every single neuron functions differently from other neurons, not just in practice, but in theory (Leibniz's law). When you sequence the RNA present in the nucleus or the soma (either one, usually not both), you will never get two neurons with the same profile. In fact, you can "barcode" individual neurons based on their gene expression profile. This profile is so complex and ephemeral that it is essentially a fingerprint of the neuron at the time you sequence it. And like I said, this profile changes very quickly over time. Just in case the connection between function and RNA sequence is not clear, look up central dogma. Pre-translational functions are a huge portion of what neurons do.

> they are all based on this basic functionality

In terms of what you mean by basic functionality, I believe you are mostly correct. However, one thing to note is that glia are separate from neurons, and there are two main subtypes of neurons depending on what type of synapse they have. You can have an electrical synapse, or a chemical one. But usually when people talk about "neurons" they are talking about the chemical type. The difference between the two are huge and this leads to my next point:

You can get as basic as you want to argue that all neurons are the same "kind". You could argue that a binary tree and hashmap are the same "kind". I could argue that everything in existence is the same kind since everything is made of elementary particles (mereological nihilism). However, my original argument is that artificial neurons are misleading when you want to think about the actual brain. Although you focused on topology (which I'm assuming you mean physical organization), I don't think you can really ignore the atomic units of the brain, which are the neurons and glia. Now we are certainly still very much in the dark about them, we have learned a lot, but we are still very far from a complete understanding. However, our current understanding is very far ahead of the "basic" idea of a neuron.

Take learning for example. A ANN learns by backpropagation. Learning is essentially just gradient descent of the loss function with respect to weights and biases. This is not at all how neurons "learn". If you focus on long-term potentiation (LTP), the strengthening of synapses, the reason why this happens between two artifical neurons is because the larger the weight and bias is in a synapse (connection between two artifical neurons), the bigger its partial derivative and impact via chain rule, meaning that backpropogation will increase its weight proportional to its impact. In biology however, it is incredibly complex and I don't really see how an ANN is really following the brain here.

Our current understanding of LTP in biological neurons is that there is short-term and long-term LTP (yes, long-term long-term potentiation). When an excitatory neuron releases glutamate (not all neurons do, a key example of how different cell types behave very differently), it binds to AMPAR and NMDAR and the receiving cell is depolarized by the influx of Na^+, which allows the Mg^+ plug in the NMDAR to be removed which allows Ca^{2+} to flow in to the dendritic spine. The Ca^{2+} binds to Calmodulin which binds to CaMKII (one of many targets of Calmodulin) which undergoes autophosphorylation but also phosphorylates AMPARS (GluRs) which results in LTP, since they are now allowing in more positive ions. However this short-term LTP is temporary, and to maintain it, it must be converted into long-term LTP. Long-term LTP requires epigenetic changes.

How high-frequency stimulation changes gene expression is not yet fully understood. But we know that G-protein-coupled receptors (which cascades down to cAMP), calmodulin, and MAP kinase all directly affect the behavior of CREB, which is a protein in the nucleus that binds to DNA and regulates transcription (a transciption factor). So CREB controls the genes that directly result in long-term LTP. One of these genes makes a protein called Arc, which travels to the synpases (look into synpatic tagging theory if curious how) and increases the amount of F-actin (not sure if this is fully understood these days) which grows the dendritic spines and therefore results in long-term LTP.

Note that what I just typed out is really just an overview, and there are entire textbooks that describe LTP. Also, I'm just a student of this topic, not an expert. And I only talked about LTP which is one of the basic subtopics of learning and memory.

So how does this compare to ANNs? An artifical neuron does not adjust its weights solely based on its connections, it must also be done with respect to a loss function. I don't think biological neurons can be grouped in a way to be said to be part of a network with a loss-function. Doesn't really make sense. The only loss-function in biology is natural/artificial selection, which means that loss is calculated at a population level, not at the individual level, and certainly not at a neural network level. Furthermore, evolution works on the phenotypes of random variation. It does not adjust genotypes to some specific goal. Brains that reproduced relatively more fertile offspring are able to slightly increase the proportion of their genes and behavior. Not sure how this translates to artifical neural networks.

What does it mean for an artificial neuron to have short-term LTP? Artificial neural networks are essentially a collection of matrices and connections. How does it go through "time"? It changes with each epoch rather than a continuous flow of time, so I guess you can make the number of epochs the analogy for time. But this difference between trial-based models and time based models also results in its own set of differences like the difference between the Rescorla-Wanger model and the temporal difference model. Being tied to each epoch and the loss function like how the R-W model is tied to each trial of CS learning means that it is limited in what it can understand and optimize for.

The way I see it, artifical neurons and networks still very far from "entering into the realm of structural topology of the brain" which I guess is another way of saying that we are very far from GAI. Furthermore, I was only talking about the disparity between biological neurons and artificial neurons in the narrow context of LTP. I'm sure I could write much more about the disparity between the organization of the brain and the organization of ANN described in the article ("topology").