What Do Tesla Autopilot and Grandmaster Chess Have in Common?

For example, here we talk about a more broad general computational approach to chess using neural networks. This is an exciting development in many ways! But also, note how it's building on already-superhuman performance of algorithms for playing chess. Part of the semi-supervised process described involves using an existing chess system (note that chess computers have for a long time been considered superhuman chess players) to assess scraped data and extrapolate synthetic data

We then want this analogy to apply to self-driving cars. So uh... let's be real for a sec. I'm not a self-driving car expert, but I am a machine learning researcher and engineer and I've worked on self-driving car agents. I've kept up with the field on and off. I've heard people like Musk claim "oh man we're gonna be there in like a year, tops" approximately every six months since 2012 or so.

This is like hundreds of binary orders of magnitude a harder problem than chess. The size of the search space, the incredible amount of precision required to be safe, and the overwhelmingly vast space of just "edge cases" that could in aggregate cause tragic fatalities and untold property damage is astronomical. Chess is a discrete state space and while not a solved game in the strict mathematical sense, certainly one we already know how to make good agents for. There is no working rule-based autopilot to simply bootstrap a neural autopilot from. I get that this is just making the same "data wins" talking point people have been saying for about a decade (both a useful insight in some domains and a convenient conclusion if you wanna sell GPUs and cloud servers), but I'm really failing to see how google throwing a bunch of compute at a more general solution to an ultra-feasible problem implies anything about a much harder and as of yet still unclear how to solve problem, realistically.

You can't solve everything by throwing more dakka at it. Attention mechanisms are the big thing right now, and they like many other breakthroughs were not just "throw more compute at it". Every big breakthrough in machine learning only does something cool when you throw a bunch of data at it. That's how machine learning works. This might mean that the well-resourced org that can afford the compute and scrape the data necessary to get something useful may not even have made the relevant breakthrough, which I guess is why they so often conclude inane things like "wow all it really took was zillions of datapoints. Scale really is everything!" No, fool, scale is necessary but not sufficient to solve problems this way. Maybe the thing that's too hard will get better with scale! But often it won't. Often we need to actually invent new types of models that capture the structure of the problem better, fiddly little details that people don't read about in soaring thinkpieces by businessmen. Sometimes we get the breakthroughs in the form of training protocols, dataset curation, or even building some boring old business logic around the "AI" in some novel way with a boring old tool that the press isn't breathlessly anthropomorphizing and hand-wringing about, like a programming language. Until someone actually solves the problem in a way that actually works, it's hard to say exactly what the solution will look like

I really don't get why people are so eager to believe, at so many times, in so many domains, that we've learned everything we need to know, and all the problems will just disappear if you just pump enough energy into it. But I do know there's a strong monetary incentive to both believe and make others believe it if you're one of the people whose stock price hinges on that belief, whose fortunes come from duping investors and governments both into funding your R&D by sheer bravado