>>Most of the time for biological intelligence there isn't an oracle describing the truth at every moment of life
This is false for most of learning of human intelligence. Most of schooled human intelligence looks like supervised learning.
Innate intelligence and other animal intelligence looks like pre-trained models and some reinforcement learning.
I think you are severely underestimating humans, and probably animals too.
People have multiple modes of learning. You can try to get an A in school ("optimize" for the objective function), but you also might be skeptical of what the teacher tells you. Likewise you might be skeptical of what you read in a book.
You can also correct the teacher based on common sense and what you already know.
Currently, a computer can do no such thing. I think he's right that there is a fundamental difference.
Humans also learn from extremely small amounts of data. (Sometimes they learn badly, but they can change their minds, or not if they still manage to navigate the world.)
What does this mean? That 18 years of schooling is a "small amount of data"? Do you think you show 2 pictures of an airplane to a child and they know what an airplane is? Do you even have kids?
You appear to be thinking of just facts and figures. AI is not that.
5-year old humans have picked up a lot. For instance, I knew about the dangers that hot surfaces, sharp objects, angry cats, and big sisters represented.
I had learned to walk, run, swim, ride a bicycle, and catch a ball.
Children learn the rules of grammar long before they realize there are any rules at all. Tenses and plurals, etc.
You would be surprised how much infants learn from listening around them and how long it takes to learn that language, and how a brain interprets language was formed through so many years of evolution to be highly tuned to learn language fast.
I understood your previous comments as diminishing the sheer amount of unsupervised learning we do as humans.
The list is almost endless. In all of those cases, if there is any supervision it is minimal.
Starting to suck when they feel something on their lips _is_ a reflex though, it isn't learnt.
Clearly there is stuff pre-programmed in the DNA and some animals e.g. are able to hold balance right after birth. But apparently most of this stuff in case of humans is instructions how to build a learning system, rather then encoding the complexity of the world in a strand of DNA. I grant you, the evolution did not have the time to encode ways to handle lego bricks and yet 2yr olds spontaneously acquire ability to manipulate these objects way beyond what any contemporary robot can do.
I'd argue that we see clear examples of such in animals that do require those for immediate survival. We ought to think about it as a consequence of natural selection rather than a 'necessity' that is somehow intelligently pre-programmed for survival. It's far more likely that path to early survival traits were mutational (accidental) rather than supervised.