> Knowledge and reasoning are inextricably interwoven in the weights of the neural network—there is no independent, explicitly represented set of beliefs.

I think an analogy would be helpful. As an LLM, reasoning through text, you wouldn't 'know' the idea of a man separately from, posterior to, the words man in say, French and English. As a human, you WOULD know the idea of man, and you would mean that idea when you say the French or English words for man.

For an LLM, although its approximation of knowledge would lead it to claim it knows they're the same thing, there would be differences in its weights that influence its usage of both the French and English words for man, and which may lead it to conclusions in one language it wouldn't reach in another. Because its knowledge/reasoning is interwoven to its knowledge, not prior to it.

> How does a set of beliefs help with reasoning?

You cannot have a syllogism without propositions.