Hacker News Reader

Comment by tocomment | Hacker News Reader

How does the brain generalize to data it hasn't seen before? Any theories?

According to Piaget's theory of development while we grow up we have different experiences from which we acquire new information. If we lets say, are naive with no experiences or memories at all, otherwise known as a "tabula rasa" stage, then we will start learning this new information and grouping it into correlated structures of knowledge, known as schemas. For example, different types of dogs can be one schema as they share characteristics and they are correlated knowledge..As we learn we not only create these schemas, but we also adapt them when new unknown information arrives. For example, if I only experience dog in my life, then when I see a cat I know that this is likely to be an animal and share characteristics with dogs, as this is similar to dogs and will most likely belong to the same or a similar schema..And that's how I personally believe we learn and interpret new information that arrives...

Of course there are many different theories, but that's my favourite.

I think in the context of machine learning the brain's ability to model the real world has evolved and a better model for the world represents a survival advantage. I don't know too much about how the brain actually models reality (and I don't know if anyone does) but the theory of machine learning still applies in the sense that each individual brain of each animal is a model and if you have a model that is too complex it will generalize poorly and therefore the owner of that brain is likely to do poorly in the real world.

It's very interesting in the sense that the totality of brains over time is essentially a sort of supervised learning with huge amounts of input data.

What are you asking here?

The brain contains/is the model. It is trained by a range of inputs and by definition it generalizes outside those inputs.

If you're asking how does the brain minimize out-of-sample error? It does that by the virtue that it's model isn't too complex for the training set, just like what you do in machine learning. If the brain had a model that was too complex it would overfit and poorly generalize just like machine learning would do with a too complex of a model...

And a follow up question, what would an example of a brain over fitting something it learned?

When we were in school, at one point our teacher switched our limit calculation questions from the almost standard notation (x,y,z for variables; a,b,c for constants) to the opposite.

Some people would have trouble handling something that had \lim_{a \to x} (some complicated f(a,x,y)) where y is a constant even though they could handle it with standard notation.

For another possible example, take something you've written recently, replace all the variable names with things like Integer, Double, and the function names with For, While (within the syntax of the language) and then try reading it.

Besides this, there's the jesus-in-toast, man-in-the-moon, face-on-mars business. The brain overfits everything, but it never stops training. It's in constant reinforcement learning.

Reply on news.ycombinator.com