Hacker News Reader

Comment by nuclx | Hacker News Reader

"Machines need a lot more data than humans do in order to get smart" - is that true?

Yes. On average the human brain just needs dozens or hundreds of examples (turns, exercises, you name it) to "learn" something. A decent machine learning model needs hundreds of thousands to millions of samples to gain good confidence and can still be fooled easily afterwards with subtile changes.

Your wording is interesting here. Do you mean humans need less data to feel confident or less data to make accurate predictions compared to a ML model? Sorry if I'm parsing your words too much here, just genuinely curious

Actually, I don't think this is strictly speaking accurate. In order to train from those dozens/hundreds of samples, human brain needs to first be developed enough by accumulating experience from billions of samples in related domains. This "pre-training" process literally takes years, and it still does not sufficiently prepare some people for some tasks.

i think it's important to note when people talk about machines vs people they always think about ideal machine vs ideal person - a rigorously educated athletic genius that can infer links between things with a couple of hours of study at most & infer future results of actions with a couple of seconds of thought.

most people aren't like this but on average engineers who are truly innovatively thinking about these problems and creating solutions are. it is just the way it is

I just think machines are good at some things, and people are good at others. If it really took billions of samples in related domains we wouldn't develop nearly as fast as we do after being born.

I'm not sure I'd call it "fast" either. For the first three months babies can barely see anything, and for at least nine they can't form anything even remotely resembling speech and can't walk. What's amazing is that all this learning is very sparsely supervised and all "subsystems" train at the same time.

This is a human peculiarity. Many animal babies are born with fully developed abilities. I am sure we have all seen the NatGeo videos of antelope babies stumbling 2-3 times and then immediately start walking and even running.

Human babies have bigger head-to-body ratio compared to all other species due to our brain being bigger. Our babies have to be born earlier or otherwise they cannot make it out of the mother alive.

Outside of that, we develop pretty quickly. As you pointed out, everything in us is developing in parallel which is quite impressive.

It might be plausible to consider the extremely large amount of sensory information humans are hooked up with. Computers generally only have very limited amount of sensory information, not to mention the inferiority of electronic sensors compared to the highly sophisticated biological ones.

I'd like to see this put to the test in a situation where sensory information is of no benefit: strategy games.

Alpha Zero got to be so strong at chess through millions of games of self-play. I would like to see how it would fare with only one hundred games, against a human chess beginner with one hundred games under his/her belt.

The general answer is yes - it's the "one shot learning" problem. But humans also have access to a large (if not very well defined) amount of data via background knowledge, which plausibly enables them to apply stronger, more effective priors to any one task.

This is the thing. Machines start on a blank slate each time, humans never.

So how do you compare the amounts of data needed?

So try it: Teach a computer about animals in general.

Then show it a single example of a new animal it has never seen, and see how well it would do at telling you if challenge images are or are not that animal.

It's not even going to be close to what a human would manage.

It's not because of the data, it's because a human understands what he's seeing.

Then you're comparing a machine that's never seen anything but animals against a human that has no separation between different kinds of data and can't start on a blank slate.

I may have all sorts of images in my mind, allowing me to make the distinction?

This doesn't sound too different from face recognition systems where you train a network to be able to tell if two faces are the same or different. I don't know what the state of the art in animal recognition is, though.

Reply on news.ycombinator.com