Many aspects of this topic deserves extensive study. For example, ML is all about generalizability, ever since the deep learning flushed the field, it seems like the numeric representation (tensor) always yields better generalizability than the symbolic representation. Is it true? Or, under what circumstances, symbolic representation helps?

In the past couple of years, several papers have shown that predefined symbolic relationship can improve over vanilla DL. For example, recognizing a picture of numerical arithmetic equations and compute the result. This is very difficult for neural networks to parametrize over the pixel space.

Moreover, statistical generalizability is currently all derived from concentration theories. This means that the knowledge encoded by a neural network model depends on the statistical distribution of the data. If two people have two different sets of data, they might end with two very different models. Symbolic generalizability is quite different. A rigorous mathematical proof holds true as long as we all live in the same world with the same set of axioms. For example, no person can appear in two physical places at the same time. Or, if A causes B, A has to occur before B occurs. These knowledge can't be learned through statistical methods with no symbolic priors. We postulate the logic first, then verified through observations.

At last, problems that statistical learning handles well so far are essentially interpolations of the collected data. Being able to extrapolate well is still an unknown. Will inductive logic programing work better in this scenario?

Symbolic is the signature of human intelligence. All of our scientific breakthroughs are encoded by symbols, even the DL (deep learning) stuffs. It won't be replaced by the numerical paradigm completely in any time soon.