a baby is still predisposed to learning language (amongst many other things essential for human living/survival) thanks to the brain and evolution. No human is really starting from scratch in any meaningful way.
And still, human experts in those fields don’t need as much data, even with our slow brains, the convergence rate is astounding, compared to machine learning.
The point you're missing is that those games have been designed by humans, for humans, so even if the natural selection didn't give us any advantage in playing chess per se, it conditioned our brain in a way that made us invent chess in the first place.
That being said, the original argument of comparing NN training data and natural selection is stupid anyway.
Leaving the details aside, the fact that a human is not starting from scratch is not in dispute. But the whole point of the discussion it seems to me is the question of exactly how humans are not starting from scratch, i.e why do we learn so much faster, and how could we apply the answer to current techniques in machine learning?
Those are still interesting questions whether or not humans and randomly wired neural nets are both starting from scratch.
It's not well understood sure but the brain is evidently playing a crucial process. Children learn to speak languages at about the same time with the same milestones occurring at roughly the same ages. Not to mention the fact that despite wildly different cultures and situations (some cultures don't attempt correct their children ever, some cultures don't speak to babies), children learn language just fine. Controversy on exactly how much aside, we're obviously predisposed to it.
>But the whole point of the discussion it seems to me is the question of exactly how humans are not starting from scratch, i.e why do we learn so much faster, and how could we apply the answer to current techniques in machine learning?
The closest biological equivalent to a parameter in an ann is a synapse. Well humans have about 100 trillion synapses. We already know that the higher the parameter count, the lower the training data required. a 50 billion parameter model will far outperform a 5 billion one trained on the same data. and a 500b one would far outperform that 50 billion one.
Economics limits how far we can go and i'm not making any declarative statements but who's to say that's not the issue ?
ann and whatever the brain does diverged in details a long time ago. It's cool to speculate and all but any special insight on the brain would have little implications on the future of deep learning. That's just not what drives architectural advances.
we could have expert level machines in a couple years but any approach trying to copy the brain is decades if not centuries away. That's how little we understand. and how little impact that actually has on the DL of today.
Current LLMS are actually nowhere near the scale of the human brain, either in parameters/neurons or training data (all the text we've ever trained an LLM on would be dwarfed by all the data humans perceive). as well as not having the headstart the human brain has. It's kind of a bogus comparison when you think about. You could easily make the case that LLMs are far more effective.
The weak spot of my argument is that it took me 20 years of on and off training, maybe 4-12 hours per day most days to get to this state. By comparison AI gets to maybe my experience level after a few years or so in months. So maybe it doesn’t actually take that much time comparatively (despite having a much lower ceiling).
The part that I’m not quite sold on though is the comparison on number of neurons. We don’t actually have a good handle on how many neurons are equivalent and a non-trivial amount of a brain’s neural net is responsible for real time signals processing of high fidelity audio and video, propiecption, motor controls etc, running your body, filtering and converting inputs into long term storage + combining it all with higher order executive functions and thought that can override a good chunk of it. It doesn’t feel like the strongest argument to make to say all that complexity is needed to create human-level intelligence in terms of comparing neurons (there may be reasons those are things are needed, but creating an LLM with the same number of neurons probably won’t work).
The compelling part for me is to continue the analogy of the brain which motivated this line of AI research. We know that the brain has all sorts of different structures and they map pretty closely to different functions and it’s not just one giant language center. Wouldn’t it make sense that we’d need different kinds of AI models to build a fully functional AI? Not least of which because specialization can be computationally more efficient (eg various computational imagery tasks are doing extraordinary things and they’re not just throwing large and larger LLMs at the problem)
A pretty obvious difference is that these models are still nowhere near as large or complex as a human brain. This network has 15 billion parameters, whereas a human brain is estimated to have 60 trillion neuronal connections. Additionally each neuron, of which a human brain has around 90 billion, can fulfill many more roles than a "neuron" in a language model.
Apples to oranges, but there's a pretty obvious complexity gap.
I'm sure there are (evolutionary?) NN models that try to do things like this but I have no idea how successful they've been.