Hacker News Reader

Comment by breck | Hacker News Reader

> It is simply not the case that I learn what a cat looks like by being shown a bunch of images of cats and having some meta-input that says "label:cat".

How do you know this? It seems to me this is likely how it works. If you spend an hour playing with a cat, and you look at the cat with your eyes at 60hz, you see about 5 million labeled augmented images of a cat.

View on HN

Intuitively I feel like there is vastly more context involved in any biological neural network learning, but particularly in the human brain. You might be inclined to reduce the learning process to just a sequence of images, but the internal reference system brings in an enormous amount of other information that can get attached: sounds, smells, touch, taste, feelings, desires and so on. These all shape not just the way the new information is pre-processed but also how it is stored and in turn that newly stored memory influences future learning. The amount of feedback loops and indirect inputs is staggering.

Perhaps I'm naive, but the difference in apparent complexity and capability between artificial neural networks as we use and understand them today, and their biological counterparts seem vast.

I'm certainly not doing research into new forms of ML and AI algorithms, but I have used the existing concepts to some degree and just intuitively it feels like what we have today is still a very primitive approximation of what nature has achieved over the course of evolution. It feels like it's not merely a question of more data and more computational power, but we are missing fundamental building blocks to actually go beyond fancy curve fitting.

Note that I'm not making an assumption that fancy curve fitting cannot eventually give rise to human level AI, but certainly at the moment I see zero evidence that it can.

I think you assume too much. Humans can identify pictures of cats and I'd be hard pressed to assume that sounds, smells, touch, taste and feel matter during that evaluation.

I'm sure you could identify a large number of animals you've only seen on film, a snow leopard perhaps.

Where does the meta-input enter my brain? How do I distinguish the meta-input from the training data?

Possibly when we learn that a cat is called a cat. Without that, we may still learn to distinguish a cat from a dog, but I guess the label would be an internal one.

For sample efficiency, there are many tests where people are shown a projection of a novel 3d object for a few seconds and then are able to pick it out from a collection of other projections of various objects (from different poses). This is not simple memorization (since a person must solve the inverse-rendering problem and then compare the 3d representations).

that's not how it works. Presuming you don't already know chinese, I can do something like show you a new chinese character that you've never seen, once, and you will be able to pick it out of a group of ten random chinese characters. Almost every time.

But how long do you show it to me for? If you show it to me for 20 seconds, you've actually shown it to me ~1,000 times.

Ok. Presuming your example is correct (it's not, neural impulses travel at roughly half the speed of sound) I could describe a Chinese character for you and you could identify it, so that's zero shot learning. Even so, if you've ever actually trained a neural networks you would know what happens when you feed the same data (n) times

Reply on news.ycombinator.com