What’s missing is interaction/causation, and the reason is that we can scale things more easily without interaction in the data gathering loop. Training a model with data gathering in the loop requires gathering more data every time the model takes a learning step. It’s slow and expensive. Training a model on pre-existing data is much simpler, and it’s unclear whether we’ve reached the limits of that yet.
My prediction is we’ll get ‘good enough for prod’ without interactive data, which will let us put interactive systems in the real world at scale, at which point the field’s focus will be able to shift.
One way to look at it is active learning. We all know the game where I think of a number between 0 and 100 and you have to guess it, and I’ll tell you if it’s higher or lower. You’ll start by guessing 50, then maybe 25, and so on, bisecting the intervals. If you want to get within +/-1 of the number I’m thinking of, you need about six data points. On the other hand, if you don’t do this interactively, and just gather a bunch of data before seeing any answers, to get within +/-1, and need 50 data points. The interactivity means you can refine your questions in response to whatever you’ve learned, saving huge amounts of time.
Another way to look at it is like randomized controlled trials. To learn a compact idea (more X means more Y), you can randomize X and gather just enough data on Y to be confident that the relationship isn’t a coincidence. The alternative (observational causal inference) is famously harder. You have to look at a bunch of X’s and Y’s, and also all the Z’s that might affect them, and then get enough data to be confident in this entire structure you’ve put together involving lots of variables.
The way ML has progressed is really a function of what’s easy. If you want a model to learn to speak english, do you want it to be embodied in the real world for two years with humans teaching it full time how the world and language relate? Or is it faster to just show it a terabyte of english?
tl;dr observational learning is much much harder than interactive learning, but we can scale observational learning in ways we can’t scale interactive learning.