Neural networks 'learn' the way humans do? A neuroscientist explains why not
theconversation.com
theconversation.com
Like, there are more than 40 separate neurotransmitters, each effectively a float output, I think? So at each synapse at every instant, potentially 40 separate floats can be communicated.
And then there’s adjacency - the network they were studying this spring they suspect is communicating using the electric fields around the neurons, so adjacent but unconnected neurons influence each other via electrical induction.
And then it’s like - and my wife says there’s no evidence neurons do this - but cells can rewrite their own DNA to store information. So each neuron is a tiny microcomputer with 40 pwm outputs and what, thousands of inputs, and 350Mib of read/write memory. And there’s 86 Billion of them in each brain?!
It just seems so much larger than a matrix of floats and weights.
But cells can affect gene expression throught multiple mechanisms, so that part of the DNA gets activated or de-activated.
You can use those mechanisms to react to the environment in various ways. For instance when you see chemical signals A and B, start making protein C. Or, when you see some signal, make the neuron more sensitive at the synapse (add receptors).
You can see it as storing state in a sort of memory, but that's pushing the computer analogy to its limits. Cells are pretty unlike chips :)
Unless you are bacteria! I will see myself out.
Of course that wouldn't mean AGI itself is impossible - biology's own mechanisms could be recreated artificially - but AGI estimates would be way off if we find out in a few years that actually we have to start over from scratch.
The problem is that it can require an exponential amount of time as well as data to actually get a decent network--and, of course, there's a high risk of it being overfit to idiosyncracies in the data. The standard approach to neural network training is to convert it into a gigantic calculus problem and brute-force it, and this doesn't always result in timely convergence or even a generalizable model.
I'm thinking the current primitive of the neuron in ML may be a bit flawed as a foundational primitive but I certainly don't think we need to model the complexities of a biological neuron.
We have a math function that defines atoms. Atoms can be simulated. If all things are made of atoms then all things can be simulated. If all things can be simulated then YES all things can be defined by a mathematical function including the brain.
In fact, it's been done already. Researchers have already simulated part of a mouse brain. The reason why they can't do humans is only scale. Humans have far more neurons so it's harder to do a brute force copy and the computing budget is more larger. See the Blue brain project, it's actually old news.
If there's something more, we don't have the tools to prove it.
This is actually not true and can logically be shown. We can model atoms with computation. Thus if all matter including the brain is made up of atoms then it can be simulated via a math equation.
The idea that we understand what physical matter really is, how it works, whether it can produce qualia, et al, is simply off the mark. Of course most physical systems can be reasoned about macroscopically and are relatively deterministic--we can understand how computers work because the functionality is almost entirely classical--and we know that quantum mechanics is both mathematically and experimentally valid, but we not at all gotten to the bottom of this.
In fact, one can argue that physics as we understand it can't be used to explain qualia. Physics requires symmetries (translational, rotational, spacetime) and asserts that there is no "privileged" reference frame that defines, say, the "objective" origin or x-, y-, z-axes. If you have two pure kilogram spheres of iron--of course, nothing macroscopic is actually 100.00% pure in practice, but it is physically conceivable and possible--they are exactly the same. One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge. This suggests that consciousness is a product not of the physical state--structural equality--but is driven by something more like pointer equality. The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
No we don't. The probability wave can be simulated. Decoherence cannot be simulated unless we have a true random number generator. But this is a simulation. We can perfectly simulate, but not perfectly recreate. In fact even with entropy for random number generation derived directly from quantum foam you cannot deterministically recreate the behavior of any atom. You can only "simulate." because the nature of the math equation involves randomness and perfectly imitating randomness involves luck.
>The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
Your statement is also highly unlikely. It's like saying the universe was created by a turtle. You can't prove or disprove it. But it can still be considered absurd.
>One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge.
The word consciousness is loaded, poorly defined and complicated. It is a language phenomenon that tricks people into thinking consciousness is some profound concept. It is not. It is simply a series of thousands of attributes related to intelligence such that if you have all the attributes you are conscious if you are missing one attribute, you are not conscious and nobody fully agrees on what all these attributes are. That's it. So they argue about it thinking it's some philosophical debate when they're really just arguing about vocabulary and trying to map out the definition of a loaded word.
It's a big trick.
A copy of consciousness would be like a copy of a computer. There's nothing magical about it. Would the copy of a computer behave exactly identical to the original? Maybe not "exactly" but they would be similar... but there is nothing profound about that.
You're getting hung up on the pointer. So if I copy a computer will the "pointer" suddenly be confused? No. It won't be because there is no "pointer" to a computer just like consciousness is NOT a pointer. It's actually shows bias the fact that you have this notion of a pointer. Think about it. If I copy a rock what happens to the pointer of the rock? When I copy a printout what happens to the pointer to that printout? Why does consciousness arbitrarily out of nowhere suddenly have a possibility of a pointer? What is it that makes you think this unprovable concept of a pointer applies just to consciousness but you don't really think about it for other mundane forms copying?
It's bias for the word "consciousness" likely caused by the "big trick" I talked about above.
You are a physical being you are not a pointer pointing to a physical being... the brain and your soul are one in the same thing. This makes sense because this is how we assume all OTHER things in the universe work. There's no pointer pointing to them so why is there suddenly a pointer to consciousness? You said it's unprovable, but to even entertain this possibility is like entertaining the possibility the universe was created by a turtle. Possible... but unprovable.
We aren't fully clear about what this model of intelligence is, but I'm instinctively sure that it won't have to involve anything as complex as biological neurons that communicate through thousands of different pathways and electromagnetic side effects too.
That's just a feeling you have.
1. We know how natural selection works, and it clearly often produces unnecessary parts for many things.
2. There is physical evidence of evolution creating unnecessary and inefficient parts and components.
Additionally the selection criteria is simply survival. Not efficiency. As long as the "processing modalities" in the brain aid in survival the selection process does not interfere. Thus all kinds of arbitrary processing modalities both efficient and inefficient can occur so long as the modalities do not detriment survival. Why does stupidity evolve? Well stupid people aren't smart enough to create nuclear bombs to kill themselves. Thus stupidity is a possible trait, similar to how inefficient and unnecessary "processing modalities" are possible, mayhap being stupider aided in survival or didn't contribute anything at all. Just looking at people responding to me on HN, alot of them are pretty stupid (physical evidence). Not pointing out which one, certainly not you.
2. There is (alot) of physical evidence of the above. For example, the wheel is a very efficient way of traveling yet no animal has evolved some form of a wheel. The reasoning is actually more complicated about why a wheel hasn't evolved but the general idea I described above still applies... both natural selection and random mutation were not able to generate the series of intermediate traits required to form wheels.
Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities".
Yes this idea did begin as a gut feeling. Subconscious processing is often times correct and logical. It is unwise to constantly mistrust it. It is always wise to analyze these "feelings" in attempt to break down the logic behind why it occurred.
What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.
For example, look at cellular reproduction. Just imagine if your phone could produce a smaller phone, which would develop into a fully functional phone. And imagine if it could do that using energy which was converted from breaking down fruits and vegetables. These are things that every living thing can do, but it's way beyond the scope of what our technology can accomplish.
There are many such miracles of biology. For instance, a gecko's feet act as an adhesive in a way that artificial imitations have not been able to match.
Wheels are also great unless they aren't. If we had evolved in an environment where we had lots of smooth roads to roll around on, maybe some animals would have evolved wheels, but instead we evolved legs which are much better for getting around uneven and unpredictable terrain.
> Thus from the reasoning above it is very possible for the brain to evolve with unnecessary and inefficient "processing modalities".
It's possible, but what evidence do you actually have that this is the case? How do you know the brain hasn't been subject to selection pressures forcing it to have evolved in an extremely efficient fashion?
> Subconscious processing is often times correct and logical.
Is it? I thought you said your brain was just an inefficient accidental mess? Why would you trust it? Or is your brain maybe incredibly powerful and capable, enough to create abstract and complicated emergent phenomena like intuition, which are so far impossible to understand analytically, or indeed to reproduce artificially?
> What my self analysis tells me is this: Given the fact that MANY of our mathematical models for physical processes involve elegant and straightforward primitives it follows that intelligence will also VERY likely be such a model as well and very likely much more simplified then the human brain.
This is a massive leap of faith and I don't know how you can possibly argue otherwise!
YOu need to explain how it's a leap of faith. That means low probability. So you need to explain the sample space and the numerator.
For me I explained. The sample space is ALL models for physical processes. Given that most of these models are elegant with efficient and simple primitives. It follows via probability that modeling intelligence by probability will most likely be the same way. See what I did there? I actually logically explained why it's not a leap of faith.
>Is it? I thought you said your brain was just an inefficient accidental mess? Why would you trust it?
First off being inefficient is different from being correct. The brain can be correct AND inefficient at the same time. It can also be correct about some things and incorrect about other things. This is obvious if you look at the world around you. People are often wrong. They are also often right. Evidence is everywhere and obvious.
> But biology has also produced systems which are far beyond what we've been able to engineer.
Sure it does. But I talk in terms of probabilities and possibilities. I am addressing the "leap of faith." I am saying because such and such is extremely possible... my statement is not a leap of faith. In this case I am saying inefficiencies are very possible. Thus my statement is highly realistic.
I never made a statement about anything absolute. I never said that absolutely the brain is inefficient. I simply said my statement is categorically not a leap of faith because it is VERY possible. And i provided logical evidence supporting this.
1. Biology can produce things which are incredibly smart and efficient. It can also produce things which are dumb and inefficient.
2. Human intellect and consciousness is one of the most complex nuanced phenomena we are aware of in the universe.
3. The human brain is one of the most complex things we know about in the universe.
4. We know very many things about how the brain processes information. We know structure and function are heavily intertwined. We know that the brain uses virtually every aspect of neural biology, from macro structures down to intracellular mechanisms, to facilitate information processing.
So we know the biology of the brain is incredibly complex. We know the output of the brain is incredibly complex. We know of many instances where the complexity of the biology contributes meaningfully to how the brain processes information. That is all evidence to support the belief that the complexity of the brain is meaningful with respect to the production of human intelligence.
It seems that you want to believe that most of that complexity is not meaningful, and your only evidence is "you had a feeling" and "biology is inefficient sometimes".
If you want to prove it's not a leap of faith, you have to present examples in which the information processing of the brain is actually inefficient, or how it can be replaced by a much simpler system. Otherwise it's all just a guess.
By your second sentence it follows that such structures aren't a leap of faith. It is very possible.
2. Human intellect and consciousness is one of the most complex nuanced phenomena we are aware of in the universe.
complexity isn't indicative of efficiency.
3. The human brain is one of the most complex things we know about in the universe.
Same as above.
4. We know very many things about how the brain processes information. We know structure and function are heavily intertwined. We know that the brain uses virtually every aspect of neural biology, from macro structures down to intracellular mechanisms, to facilitate information processing.
But we don't know whether the entire structure is efficient.
>That is all evidence to support the belief that the complexity of the brain is meaningful with respect to the production of human intelligence.
But it is not evidence supporting the thesis that the brain being inefficient is a leap of faith.
>It seems that you want to believe that most of that complexity is not meaningful, and your only evidence is "you had a feeling" and "biology is inefficient sometimes".
The feeling has been rationalized. I explained the logic, it is now backed up. The criterion here is not absolutist. The criterion is just stating whether or not such a scenario is likely. IS it a leap of faith or is it not? I've sufficiently proved that the brain being inefficient is clearly NOT a leap of faith.
>If you want to prove it's not a leap of faith, you have to present examples in which the information processing of the brain is actually inefficient, or how it can be replaced by a much simpler system. Otherwise it's all just a guess.
Easy we have many ML models that already beat humans at certain tasks.. Those models use much less neurons then the entire brain itself and follows a much simpler model. See spam filters. It shows that a simpler model is already possible for certain aspects of intelligence.
I disagree with you. I believe I've shown enough evidence to move the probability out of "leap of faith" territory. I just threw in the example up above because you requested it.
> There must be something "turing complete-ish" about neural nets. As in although the primitive element of a human neuron is complex, you can achieve identical computational power with a much simpler model.
So the assertion is, that you can achieve identical computational power to a neuron with a much simpler model.
That is a leap of faith. There is insufficient evidence to conclude this. If you said "I believe there might be" or "it's possible that" I would not have taken issue with it, but there is a very high burden of evidence to assert this belief. Without that evidence you are operating on faith.
Until we can successfully reproduce something approximating human intelligence, or until we can postulate a credible mechanism by which human intelligence arises, it's a very very big leap to assume we understand anything about what is or isn't required to produce it.
> complexity isn't indicative of efficiency ... But we don't know whether the entire structure is efficient.
We don't know the complexity is efficient, and we don't know whether it's required. What we do know for a fact is that a human brain is capable of producing human intelligence. We don't have evidence that any other system is in fact able to produce human-like intelligence.
> But it is not evidence supporting the thesis that the brain being inefficient is a leap of faith.
I don't have to produce evidence that it's a leap of faith. You made a claim that is not supported by evidence. You have to provide evidence for your claim, or else it is defacto a leap of faith.
> The feeling has been rationalized. I explained the logic, it is now backed up. The criterion here is not absolutist. The criterion is just stating whether or not such a scenario is likely. IS it a leap of faith or is it not? I've sufficiently proved that the brain being inefficient is clearly NOT a leap of faith.
Yeah but you can rationalize just about anything. I could provide a rational argument for why God exists. It's still a leap of faith to believe in God.
You have a rationalization, but it falls far short of being any kind of rigorous proof.
> Easy we have many ML models that already beat humans at certain tasks.. Those models use much less neurons then the entire brain itself and follows a much simpler model. See spam filters. It shows that a simpler model is already possible for certain aspects of intelligence.
All ML models prove is that we have been capable of producing machines that are capable of pattern recognition. It's a massive leap of faith to generalize this result to assert the belief that neurons in general can be modeled at a much lower level of complexity. It may be the case, but there is insufficient evidence to conclude that. It's just your belief based on gut instinct, and the fact that you've talked yourself into it.
I use the word "Must" as in "highly likely" if my words didn't convey the probability then my bad, but I assure you now that my intent is conveying a probability.
> There is insufficient evidence to conclude this.
There is enough evidence to show it is sufficiently high probability for consideration.
>If you said "I believe there might be" or "it's possible that" I would not have taken issue with it,
I did say this MULTIPLE times in subsequent responses. I am telling you now that is 100% what I meant. Now that we're clear, we are in agreement. This conversation is over.
Agree to disagree.
It is still proof by contradiction, what you say is categorically not true.
And the behavior of those receptors might also be modulated by other signaling mechanisms inside the cell: i.e. neurotransmitter A might modulate the signaling of neurotransmitter B.
In other words, there is a lot of complex information processing going on there.
The useful part of AI is the inference/output. If we achieve human-level performance AI inference then we have an incredibly powerful tool that can revolutionise society. The learning algorithm, be it gradient descent, evolutionary mechanisms or something entirely new, doesn’t matter. Whatever gets the job done.
If we create a neural net that has the equivalent knowledge and intelligence of a human 5 year old but that neural net learns slower and was produced on 10 trillion pieces of data then we STILL have created a sentient AI despite the fact the learning algorithm is shit.
Which doesn't stop neural networks being immensely useful (and even significantly more useful than humans in relevant domains, especially five year old humans), but might well stop them from being sentient. Neural nets are probably already better at passing Turing tests than five year olds, but the five year old's love of mummy doesn't come from statistical analysis of a corpus of texts about love and mummies.
I would soften the idea a little bit by noting that as a little human grows up and learns more things, its brain plasticity goes down over time. AFAIK there’s no evidence that learning “saturates” the ability to learn other things, and the loss of plasticity is probably mainly linked to aging and/or other factors, but functionally the effect is very much the same: humans that know a lot of things typically have less capacity for learning.
https://www.youtube.com/watch?v=ZeiMhUlipTk
These people are clearly still sentient.
There is still some form of memory there but the overall mechanism is largely gone.
I don’t think we need to emulate every connection, every neuron, or every circuit— but I do think the broader neuroanatomy (this region mainly processes inputs from that region, and has these feedback loops, and then mainly has outputs to this other region) is essential to emulating the way brains work.
I may very well be wrong, and I hope people in the field can forgive my insolent impression, but my impression as a biochemist/engineer is that people in the comp sci world and people in the neuroanatomy world don’t really speak the same language: one is usually bad at the other, so there isn’t really a tight transdisciplinary coupling of forces.
I think mathematics is one field where AI could be actually immediately useful.
Sorry for laughing, but that's a rather funny thought to be given incorrect maths lesson via AI à la college professor.
The field of AI-applied to mathematics is already somewhat developed actually!
At the moment it's primarily done through proof assistants such as Coq[1], but there is work done to go further and apply ML to the field [2][3]. If you have any interest in the field, I would highly recommend you to take a dive and even play around with proving some simple theorems using Coq. This article[4] from Quanta is a good start to read a summary on the background and work up to now.
[1] https://en.wikipedia.org/wiki/Coq
[2] https://resources.mpi-inf.mpg.de/departments/rg1/conferences...
[3] https://arxiv.org/abs/2104.05207
[4] https://www.quantamagazine.org/how-close-are-computers-to-au...
They do. The GPT family is basically a very fancy autocomplete, so it’s mainly noteworthy because of how far something that limited can go, as it’s not even trying to be a fully general NLP system.
If you want to solve maths problems, there are many better choices; I often use WolframAlpha, which is great for the maths part of my problems, but fairly often needs me to try several variants of my query because of the limitations of its natural language processing.
Personally I think the ability to truly "do mathematics" may be a marker of AGI. If/when we have machines independently mapping out new and interesting areas of mathematics the way human mathematicians do I suspect we will have cracked meaning/understanding. FWIW I've felt that way for over thirty years and have never considered the Turing test a convincing marker of intelligence, so I don't consider myself to be moving goalposts when I find the (absolutely incredible) feats of the current generation of ML algorithms unconvincing as evidence that we're nearing AGI.
It's actually not hard to write a program that can do what he does.
"What is wealth? Why are there unequal amounts of it? It's because wealth is what people want. If you can figure out what people want, you can generate a lot of wealth. Some people don't know this. Some people aren't cut out for startups. But people who know what people want can generate billion-dollar startups in record time by the standards of our ancestors. Unfortunately, there are people in politics and government who do not understand that wealth is what people want, and therefore cannot be regulated, because to do so is to regulate what people want, and companies that operate on this principle will find themselves unable to hire great programmers."
This stuff isn't hard to generate... I did it by hand, stream of consciousness.
e.g. https://twitter.com/sparuniisc/status/1377333798442967041
[1] And the cited Nature paper does not seem to say exactly what the article says.
Imagine a program written with OOP, and the same program written with FP. The implementation details may be rather different, but this doesn’t prevent it from doing the same thing in a different way. You have to think of artificial neurons and real neurons in a similar way (though they are rather more dissimilar than OOP and FP are to each other).
The main problem that backpropagation solves, I believe, is roughly equivalent to feedback loops in real neural networks.
Back propagation assumes a synapse can be modeled as a weight times an input. We know of tons of types of synapse which don't work like that.
Because you’re pointing to the first, whereas I’m pointing to the second, and the article points to the second as well.
I agree with you that not all arrangements of real neurons look like backpropagation: neurons are way messier and more diverse in their configurations and in the way they interact with each other.
If your point is, that something approximating back propagation probably happens somewhere in the brain, then I would agree with you. But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all.
And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
Agree generally
> messy no
Well, the way I originally meant it was that it's a very high-entropy, chaotic organ, similarly to kidneys. There is order in the chaos, obviously, but the fact that it can work at all is very remarkable (just like kidneys, where you have messy and complex chemical soups and active concentration gradients all over the place in every which way). In fact all of biochemistry and cell biology, generally speaking, is a very remarkable juxtaposition of chaos and order together and at the same time.
My area of research was studying cell-derived extracellular vesicles (of which you can count viruses, extracellular mitochondria, exosomes, microparticles, apoptotic bodies and others), which are a vehicle for the horizontal transfer of information (adding a whole other dimension to cell signalling). Studying this is a great way to appreciate the amount of messiness and chaos we're dealing with... both in pathology and in physiologic conditions :)
> But I think the broader point of the article is that there is no reason to believe that the function of ANNs has much to do with that of in vivo neural networks at all.
I mean, I agree that artificial neurons don't look like real neurons, but I prefer to think of the higher-order function, at the level of the organ, or group of neurons, where you get emergent behaviour and where meaningful computations happen. I think the strongest analogies are found when you relax the rules of what you're comparing (to find the most similarities, we should compare groups of artificial neurons to groups of real neurons).
> And there's no reason to believe the current approach to ANNs will result in anything like human intelligence, which I think is often an assumption made by observers of the field: that ANNs essentially just have to be scaled up to achieve human-like intelligence.
Strongly, *strongly agree*. I believe if we want to create a computational network that resembles the way a brain works, we need to start modelling our networks like brains at the macro-level (not on a neuron-to-neuron basis of course, but on a neuroanatomic, group-of-neuron basis).
When's the last time you've heard of AI researchers modeling a spinal cord, or a pons, or some thalamic nuclei, or specific neocortical functions? I think if you try to compare a layered sandwich of artificial neurons to a brain, you're necessarily bound to find all sorts of important differences in performance or function. Neuroanatomy has a lot yet to learn, but the structure-function relationship is a lesson that was learned a long time ago.
And to be clear, in brains there are very few structures that are self-defined and not connected to a whole lot of other inputs and outputs... So while the group-to-group comparison may be broadly similar at the core, the comparison will break down at the boundaries. In other words, the brain is a very side-effecting computational structure, and it is difficult (if not impossible) to isolate a purely-functional set of neurons with clearly defined inputs and outputs.
So machines, for now, use two tricks to get a bit better in maybe more narrow fields. There is NNs without back propagation and unsupervised too though, they are just currently not the best, right?
Doesn't sound fundamentally different to me.
And about this paragraph:
"humans learn by making structured mental concepts, in which many different properties and associations are linked together. For instance, our concept of “banana” includes its shape, the colour yellow, knowledge of it being a fruit, how to hold it, and so forth.
As far as we know, AI systems do not form conceptual knowledge like this. They rely entirely on extracting complex statistical associations from their training data, and then applying these to similar contexts. "
That's plain wrong, isn't it? I'm no expert, but isn't latent space and layers basically these "structured mental concepts"? You have upper layers encoding for partly features, lower layers checking on many of them. That's abstract concepts. And you have the multidimensional statistics of the latent space that will see bananas as the intersect of yellow color and a long shape.
For an opposite "opinion": the layers of a NN map quite nicely on to what happens in the brain: https://twitter.com/JeanRemiKing/status/1533720262344073218 (they check which neurons get activated first, which later in NN and brain)
Kid touches hot stove. Gets feedback. Kid tries walking. Gets feedback. Kid talks to someone. Gets feedback.
I have hard time even coming up with an example of a proper unsupervised learning in the wild. Hermit phenomenologist, maybe? You have no way of doing anything social in an unsupervised way and most of the language happens socially.
EDIT: I did misremember RL as supervised, see chain below.
EDIT2: Hey, I never said supervised in the first place! Enough internet for me today, I need a nap.
When a kid touches a hot stove, they classify it as bad/dangerous through feedback - unsupervised learning. Not because a parent comes by later and says "Hey! Look, your hand is burnt! That's bad because I say it is!" - supervised learning.
You can even see supervised learning in kids to some extent: when they play barber shop and get a butchered hair cut, you often see kids be perfectly fine and happy even with the outcome, through lack of supervision they don't yet have any negative experience from doing this. Then a supervisor comes along (parent) who freaks out and the kid learns through supervision "oh no! I did something bad because mom said so!"
Let me start by saying, I misremembered and believed RL is considered supervised. Since it's actually not, that dramatically changes my position ^^
I'd argue then that most learning in humans (how strong analogy holds aside) is RL. The world is structured so that you learn to avoid pain - those who don't tend to not leave a lot of offspring behind.
Arguably that's a better metaphor for all learning than supervised/unsupervised: there is some further reward/punishment for even the school work. Misuse of muscles when learning to walk is also connected to similar feedback, so is misuse of language - be it grammatical failure or being a bad salesperson.
Generally I'd focus on pointing out that children do not learn in an unsupervised fashion in almost any setting, claiming it was supervised was wrong and based on my misclassification of RL.
It’s quite concerning too that there seems to be this belief that one can be effective in this field with a weak math background. I recall a meeting with a consultant claiming that they could improve order flow by using ML with a data set that was small and inaccurate. Their background was an mba and a few years of working on mergers and acquisitions.
A large portion of the population is being told that programming is where you go to make a ton of money, and then subsequently hear that machine learning is the future of programming where you don't need to understand your data, you don't need to understand anything. You just plug in some magic codes and get out great results because the computer learns what you want!
Of course, it's not like that. But we're going to see a lot more of this attitude, because Machine Learning is not pitched as "high level statistics" but rather "magic that does what you want!"
As for the physical parts: isn't that a loss function? Reward/dopamine if you reach your goal (walking), pain if you do what the rules/the body does not allow.
I'm sorry if I'm mixing terms a bit, as said, I'm not an expert and a bit rusty about what I once knew :)
There is quite few animals that can do a lot instinctively, right? Like a turtle hatching finds water, swims, eats.
Also reminds me of the experiment with rats that genetically transferred fear for a specific smell. ( https://www.newscientist.com/article/dn24677-fear-of-a-smell... )
What you are referring to is an internal representation of the compositional nature of the input; the NN will holds a structure of patterns that represent the idea.
But the "structured mental concepts" in question are entities interesting for their logical manipulation upon reflection. Whether some NN definition (configuration) is apt for the latter should be seen in view to the latter.
Well, this has been talked about for at least 10 years. Just take a look at any talk/FAQ by Hinton and/or LeCun from 2012 and onwards.