Hacker News Reader

Comment by TheRealPomax | Hacker News Reader

It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience. A human doesn't come to minecraft "knowing nothing", like the machine learning programs in this challenge. They already know basically everything there is to know in order to perform task-oriented goal planning and only need to focus on task discovery.

If anyone had pass the bar this challenge set, they'd have basically solved the biggest hurdle in general AI, and would probably have received a call from John Carmack.

View on HN

That's kind of the point. The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge. For example, DeepMind's last three or so iterations of their Alpha-X architecture for game playing did not even have any prior knowledge of the rules of the games they tackled (chess, shoggi, go and Atari games) and were trained entirely through self-play and without anything like an opening book of moves etc, like the origional AlphaGo architecture (or DeepBlue and traditional chess engines).

That is very big in AI right now, to try and remove the need for the "human in the loop" and learn complex tasks from 0 background knowledge.

Obviously, like you point out, this is very difficult to do and it requires lots and lots of data and compute (no, more than that. Really lots). As the article reports, this eliminates any hopes of "democratising" machine learning and makes advancing the state of the art a game that only large corporations can play with any hope of winning.

But there is a lot of "common knowledge" that's useful

All the problems solved by fine-tuning image net for example, instead of training from zero

Identifying objects in a 3D environment is useful across several domains and it shouldn't be limited to Minecraft

>The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge.

That's not a very useful goal. I can trivially make a game that is impossible for such an AI to win. It would require you to enter a 1024 bit code [0] and if you are correct you win. For a speed runner the time it takes to beat the game is merely limited by their typing speed because the code is always the same but an AI with no prior knowledge can only brute force the solution. How does the speed runner win in the first place? They ask the game developer and then post the code on a wiki.

[0] an euphemism for minecrafts crafting system

What if you had 22 hours of footage showing speedrunners entering the 1024 bit code to train on?

It would be possible to do that, but it's not really what was asked.

The way you'd train a neural network with that footage is comparing the recording of the neural network playing the game to the recording of the speedrunner, and scoring it based on the difference in the footage.

However it wouldn't really be learning "from scratch" and "end-to-end" anymore, since you're essentially teaching it to imitate and play like a human player. Also your network would likely be just blindly executing a series of learned actions, without any real understanding of the game. If you confronted it from some situation that's not part of some speedrun it memorized, it would probably be at a loss. Pretty useless.

How about it watches how the human obtains the secret number (1024) in this game, then next time extrapolates the process to a different game with a different secret number? That is the kind of learning that humans do and AI's can conceivably do the same.

That's just not how neural networks learn. They generally can't "watch" others to learn.

In the example above, at no point the neural network would be watching the video footage. The reference footage is just used for (automatically) scoring how well it does during training. It's not fed into the network itself in any shape or form.

The idea is that such end-to-end techniques can be used to tackle problems for which we don't have background knowledge. For example, (according to DeepMind's latest paper on MuZero) real-world robotics tasks etc. Atari games are a good use-case for this because they dont' have very clearly defined rules like board games have, so a model basically has to first learn to play the game before it can learn to play it well.

You can perhaps design a game that is difficult to solve by AI (though I wouldn't bet on it). The question with AI is usually not what it can't do but what it _can_ do. If a certain system fails at your task but performs well in a range of other tasks that are considered useful and challenging, then it's worth consideration. Especially so if there is no system that can perform well on that one task.

You are confusing the goal and the method.

Setting up a stupid method of getting to the goal doesn't disqualify said goal.

> The trend in machine learning right now is to try and solve problems "from scratch" by training "end-to-end", which means without any prior knowledge.

In a strong sense, this implies ML is not "intelligent" in any conventional sense of the word. What little we have of understanding of learning and intelligence in the scope of humans strongly implies that drawing parallels with solutions to other tasks, building on previous knowledge, is the key to intelligence.

Perhaps the purest expression of this is in pedagogy of math. Piaget and Brissiaud are some of the scholars who I think really grokked this. They brought forward the concept of compression as essential for learning. E.g. if you have learned but not yet compressed the operation of addition, you will have a very hard time learning multiplication. After addition has been compressed such that you don't have to spend any effort doing it, you are ready to learn multiplication.

Maybe it would be worthwile to explore if there are AI techniques that can learn simple tasks first, then use that knowledge to solve harder tasks?

But they do build on previous knowledge (that's the training part), it's just the such knowledge is built from scratch, rather than taught by others.

A human can learn something through a fairly small amount of specific knowledge via written, verbal, etc instruction.

All the data and the training process together are very clunky in comparison.

IE, an inability to be taught by others is a serious limitation to say the least.

Perhaps we can learn from small data, perhaps that’s an illusion. How much of our mind is hard-wired by our genes, our genes being a very slow-learning form of intelligence?

Or, when we see “one” example of a new animal (for example), do we see a single picture? Or do we see it moving for about a second? Effectively multiple images? And, having previously learned a general model for physics, stereoscopic vision, and lighting, does that one second of motion gives us the size, shape, and a guess at the musculature of the animal?

If you strip those advantages, how many training examples do we need? The examples of people blind from birth or from extreme youth who regain vision in adulthood[0] implies that we learn a lot in our youth about how to see, which we can share between recognition tasks we learn later. Keeping those basic skills — avoiding “catastrophic forgetting” — still seems to be a rare and advanced feature in AI, but it’s not impossible.

There is still a lot we don’t know about how our own minds work. We might be closer than we realise, or much further.

[0] https://en.m.wikipedia.org/wiki/Mike_May_(skier) amongst others. I never did find the name of the guy who learned to see from touching a monkey statue…

It's certainly worth keeping in mind the limitations that you describe and human's learning activity is certainly not as the rationalist intuition. But to take these limitation as entirely characterizing human activity seems bend the stick too far the other way and fly in the face of the many ways humans have modified their environment and adapted to those modifications and fairly often learned with only a few examples. And sure, plausibly a fair portion of those quick adaptation involve reworking basic facilities like vision. The whole thing is consistent with the neocortex sitting on top of the rest of brain and serving as a means to rewire it.

> A human can learn something through a fairly small amount of specific knowledge via written, verbal, etc instruction.

This demands the mastery of language which is a very high bar. Most animals haven't achieved this, so I think if you expect to get a computer to learn this way before being able to reason as well before you consider it having intelligence, you would also have consider most animals not having intelligence as well.

> This demands the mastery of language which is a very high bar. Most animals haven't achieved this

Then avoid language as medium. Many animals can learn from being shown an action once and apply it to themselves.

But it's not transferable. If you teach the AI to play checkers, you don't use that knowledge when trying to teach it chess.

Tranfer learning is a form of this, where you train the first half of a neural net over a huge set of data, then take a much more specific task and train just a bit more by adding some layers, and you get great sometimes industry leading results. Look up Google's BERT project, trained generically on English texts and then you can reuse that part in many different tasks like sentiment analysis or entity recognition or translation. An amazing way to leverage Googles compute and data access and do cool things even as a single developer.

> as a single developer

I think you are out of luck already. Even with transfer learning, retraining BERT to fit a larger dataset takes ages even on an RTX 8000. A single state-of-art experiment/training run at Facebook now costs 7 figures. The age of individual SOTA ML researcher/developer is over.

This is so sad, actually, and It's not just ML, it is every kind of new tech and discovery.

To me it shows we have already picked the easy fruits and technology will not continue to evolve exponentially.

> In a strong sense, this implies ML is not "intelligent" in any conventional sense of the word.

I don’t know why people have so much faith in ML and AI. Considering what a CPU actually does, shouldn’t the default position be one of skepticism?

It’d be truly amazing if you can make a machine intelligent in the human sense by merely coming up with a clever algorithm.

Or we could flip your question around:

Considering what a neuron actually does, shouldn't the default position be that the brain is a computer?

How certain are we of what neurons and other building blocks of the brain actually do?

No, because you’re being reductive. Biological organisms are clearly not like silicon computers.

Maybe biologist come up with the next big break through. I mean, if you want to match an organically grown organism ...

Is it clear that they're not alike? In the end it's all information and data being processed, whether by silicon or neural cell. What difference does the medium make to the end result?

Then at the very least, you should be able to simulate a biological brain, neuron by neuron, and get an intelligent computer that way. If you agree that's possible, you also have to consider whether it's the most efficient approach -- plausibly special algorithms could be invented that get the same intelligence at a fraction of the computing power.

Every advanced tech has been used as a metaphor for how the mind works. People used to think the brain worked like a piston engine. Now people think the brain is like a computer. Some people also think we are a network ala the internet of biological persons. Lots of artists think in images. Some people see the brain as an evolving story. The people who have any idea how the brain works are those that have been studying it for decades, I would be interested in those metaphors.

The conversation around artificial intelligence is more of a call to action for human intelligence to figure out what we need to do and how wield ML than it is a description of the fundamentals of the computing. Even if the results are mediocre in the extreme, you can bunch together a huge amount of social and business change under that banner. I'd argue that's why people have faith in AI not because of what a computer can do, but because it is a call to faith in humans ourselves to integrate the technology and all implicated changes to it's maximum potential.

They do build on previous knowledge. That’s literally what the training is doing.

It’s just easier, engineering wise, to start over from scratch on each iteration. And it makes it easier to evaluate how each change improves the outcome or not.

> with only 4 days of training data

so 4 days to train their AI and not 4 days of training data.. on the data which was "60 million frames of recorded human player data"

My bad: they got 2 weeks of footage. Other than a clerical error, of course, this changes nothing about the fact that that's nowhere near enough data unless you've already cracked general AI.

>It's a weird thing to somehow think that an AI with only 4 days of training data should somehow outperform even a human child, which has a decade or more of imitation learning experience.

Well...not so much if you consider that machines are orders of magnitude faster than humans in any other task, including processing and recording data, a.k.a. learning.

Not really, no. Procesing and recording data is not "learning". Machines are incredibly much faster at _classifying_ but that's been our problem for decades now: we figured out how to make insanely efficient and unreasonably good classifiers. However, we haven't been really able to turn that into efficient "learning", the best we've been able to do is what Deep Mind does, which requires very power machines working with literally decades worth of data.

> with only 4 days of training data

You misread it. It's around 300 hours of training data. It's 4 days to absorb those 300 hours.

I bet a human child who has never even seen a computer could learn to do this in far less than 300 hours. I suspect even 1 hour would be enough.

60 models trained on one dataset as well.

Now we just deny the prophecies of the upcoming singularity?

Reply on news.ycombinator.com