> It requires a theory of mind, which means understanding human goals.
Like chess?
Like chess?
Chess is really just about searching through as much of the space as possible, and you can take a few seconds. And believe me, there's a whole lot more stupid behaviours you can encounter on the road while you have to react in less than a second.
That isn't really an accurate description of how the modern DL bots work. They don't need to reference any database of past games while they play.
This reminds me of the kinds of intuitions people had about Chess and Go. In retrospect they seem silly, but it made plenty of sense to them at the time. The fact was that there was a solution that machines could use that humans couldn't use. Naturally, a solution that humans couldn't use was hard for them to anticipate being effective.
As an experiment, try making a chess AI without explicitly giving it the rules of the game and see how it performs.