I think there is a lot of merit to this approach. Ultimately we live in a world guided by physics and macro-level perception driven by our senses and our own motor control. Of course newtonian physics is not the end all be all -- cell biology or quantum mechanics works on a very different level... but what is important here is that we know that human beings understand these things and make novel discoveries on these things using a thinking apparatus that was pre-trained on large scale newtonian physics. I've always found that even in advanced mathematics my mind always uses low level geometric analogies. So the "embeddings" or priors that can be obtained are probably much better than what can be done through text correlation as with LLMs. It's very different to learn the word bounce through observation of a physical model of a ball bouncing vs. seeing what other words it co-occurs with.