The "AI" came up with a slightly off number and the researchers struggle to understand why. They struggle because "the AI" is a black box that lacks explainability and cannot produce an explanatory model, only a predictive model [1]. The simplest conclusion is that their "AI" has not learned any useful models, and has only learned to accurately predict their test set, to which the authors had acces throughout the training of "the AI" [2]. The simplest explanation is that their model isn't very good at doing what they wanted it to do, but they opt to explain it as a scientific mystery, instead. Why?
It seems the only reason to try and see a scientific mystery where a simple failure would suffice as an explanation is because the model is "an AI". There seems to be some kind of expectation that "an AI" must have some deeper understanding of physics than humans, even when we don't understand what it's doing, even whe it's not doing very well.
In short, this seems to be based on very wrong assumptions and to be coming up with very wrong conclusions, but then again it makes for a great headline and so here we are, discussing it on HN.
____________________
[1] To clarify: a predictive model is one that can predict novel events. An explainable model is one that can explain how it made a prediction. An explanatory model is one that explains how the world works and why certain predictions are true, or not. Predictive and explainable models are useful, but most scientists aim to build explanatory models, because an explanatory model is necessarily also explainable and ultimately better at predictions, while a predictive model is not necessarily explainable or explanatory and an explainable model is not necessarily predictive or explanatory. Deep neural nets can only build predictive models, which are sometimes explainable, but they can't build explanatory models. That's because they can only identify correlations in data, but not explain those correlations with generalised theories, based on previous knowledge.
To put it plainly, neural nets can't come up with scientific theories, but they can estimate probabilities of things happening. But scientists can and want to come up with scientific theories, not just predictions.
[2] When the experimenter has access to the test data and can tune the learner's model until it scores highly on the test data, that leads to a model that overfits to the test data. Such a model is useless for prediction over unseen data (i.e. data not available to the researchers during training).