Although LLMs are incredible feats of engineering, they're useless scientifically. The hallmark of a good scientific theory is that it not only explains what's true but that it fails to predict what's false.
There are constraints that all human languages obey [1]. Humans are incapable of learning languages that violate these constraints (i.e. we don't have hardware acceleration for them and are reduced to explicit symbolic manipulation). However, LLMs are just as capable of learning inhuman languages as human ones, so they tell us nothing about the nature of human intelligence, or at least our language capacity, which is our most distinguishing feature from every other species on earth.
[1] This isn't an example of such a constraint, but it's fun example of human limitation: center embedding! We seem to be incapable of doing it more than once or twice. "A man that a woman that a child that a bird that I heard saw knows loves" is perfectly grammatical but nearly impossible to understand without seeing it in print, whereas we can right embed all day long: "a man who is loved by a woman who is known by a child who was seen by a bird that I heard".