Comment by ItsBob | Hacker News Reader

Comment by ItsBob | Hacker News Reader

Not to derail this conversation but...

When I'm explaining AI stuff to family, the example I use is classification and I specifically use cats and dogs. I use the analogy of how you teach a toddler that this is a cat and that is a dog. Essentially repetition. And at first they get them mixed up and the parent will say "no, that's a dog" when they think it's a cat and so on.

But essentially, for a child learning the difference between a cat and a dog you only need to show them a handful of each and they'll generally get it from that point on.

That being said, why does it take ML millions (or billions) of images to be able to say "that's a cat" when a human does it on a handful (might be up to, say 100 but my point stands). Why can ML not do that yet?

I'm a dev for many years but not in AI, hence my ELI5 question :)

Edit: If the answer is massively long and complicated, perhaps if you could point me to some text (book, paper etc.) and I can read at my leisure.

Edit2: I just thought of something. Is it related to whether the child sees a still image or a live cat? So, for example, a still image is a single example of a cat standing in a particular position etc, whereas a moving, live cat, would be interpreted by the brain as many many still images, all processed individually? The end result being that, in fact the child, when seeing a live cat, actually sees thousands or millions of still images of the cat? It just popped into my head there :D

Saying "it takes a hundred images for a human to learn" implies that you can take a baby/toddler/whatever, who has been blind all their life, restore their sight, show them 100 photos of dogs and cats, and expect them to know what's what.

You're ignoring the trillions of frames a toddler has seen before they get to the part where they can even understand what a photo is.

Toddlers have a neural network architected and fine tuned for object recognition. They also have a benefit of learning thousands of hours of video from the real world, initially blurry, gradually getting sharper, to draw the model of physical reality before they are given first examples of "dog" or "cat" to classify. If you kept toddler immobile in the darkness then presented him with static 2d photos of a dog or a cat I think it wouldn't learn much faster than artificial neural net.

Neural networks do not imitate the brain and “training” a network has nothing to do with learning, despite these terms serving double duty.

Machine learning models are just math functions fitted to some data. When we get predictions from them, we’re really just using a technique to interpolate between the data points. The denser a particular region has been sampled in the data, the better the predictions will be. (This is why GPT-anything will do a good job writing solutions to common leetcode problems, while struggling with a novel problem.)

Humans have a powerful abstraction ability far beyond any algorithm that has been developed. We can take in a few pieces of information describing a really unusual set of circumstances, run imaginary experiments and simulations on them, and make very granular and accurate predictions about their consequences. Nobody actually knows how.

> just math functions fitted to some data

I don't want to be pedantic. But I'd like to interject. We can simulate everything with math functions. If the algorithm isn't there yet it's because we're using the wrong functions.

Are you a physicalist or a dualist, in the philosophy of mind senses of those words? I don't see how what we do is much different than computers as both use fundamental physical computation to achieve a result. It could simply be that our brains are much more complex in their computation than current computers, but they both compute nonetheless.

I have no idea! I don’t think these questions can be answered (despite modern culture expressing absolute faith in physicalism).

But back to the subject of deep learning, whatever brains do, I think it’s pretty clear that existing neural networks don’t approximate the biological process. They’re just too static.

1. Children are not tabulae rasae, they have previous optimization of their brains via evolution such that they are much more predisposed to visual and other human tasks than computers. On the other hand, computers are tabulae rasae such that they are essentially blind and have no optimization until you train them.

2. Children see many more data points than computers, thousands of hours of lived experience from which they can learn, of not just the visual world but the auditory and other sensory worlds.

Essentially, you are comparing something that is ~80% trained and then training it on billions of data points to something that is 0% trained and then training it on still images of only two dimensions and asking why the former is much better.

Millions of years of evolution has trained the biological LLM in our brains to be good at fine tuning those concepts.

Reply on news.ycombinator.com