If prediction cannot replace reasoning, then how would you define reasoning? If reasoning is the process of inference of the most likely from the most similar known, then how does the multi-head attention transformer not fit that description?
I don't know, because I did not propose that definition of reasoning.
"Reasoning" does not have a formal definition in AI, or Computer Science (neither does "intelligence" or "understanding") but, in general, when we speak of reasoning in those fields we mean a procedure that can derive the logical consequences of some premises, often in conjunction with some background theory. I'm happy to use this as an informal definition of reasoning, if you want.
I wonder if expert systems could be used to generate a "logical reasoning" training dataset (or a cost/fitness function) to help train/evolve neural networks on. Or if there are other ways of integrating these two.
So, how do you decide whether something is logical? In particular, if we have some system S that exhibits behaviour H in context B, can we tell whetehr H is, in some sense, "logical"? Why, yes we can: we can apply the rules of whatever logic we think S may be following with H and see if we can reproduce H in the context of B, starting from the same premises as S.
For example, if S is producing a behaviour H that can be described as {A → B ≡ B → A} and we think that S is trying to reproduce propositional logic, we can say that it's failing, because H is not correct by the rules of propositional logic (to clarify, H could be something like "If it rains it will be wet therefore if it is wet, it has rained", which is not correct).
Of course the problem with GPT-3 and friends is that they are not trained to output statements in a formal language (at least not exclusively) so it's very hard to formalise the logic, or lack thereof, of its output. Though it would be interesting to "hit" GPT-3 with some prompts representing the beginning of natural language versions of formal problems.
Could expert systems be used to generate training data for GPT-3? Maybe. If an expert system could generate good quality natural language, even only natural language restricted to some limited domain. Then yes, why not? You could probably train GPT-3 on its own output, or that of its predecessors, as long as it was curated to remove nonsense (which is much harder than it sounds).