Given that we are able to prompt GPT (davinci) to learn proprietary DSLs that it would have never seen and build programs with it on data it has never seen… I find it really difficult to buy that there is no internal representation of logic in the model hidden states.
GPT-3 has seen examples of code with comments describing a DSL followed by use of that DSL. And DSLs rarely invent a new paradigm. I think if you showed a model OOP programming when it had only been trained on functional code it would fail spectacularly. But your DSL is probably not far off from Lisp or C, or YAML. The model correlates a pre-defined set of tokens with other tokens in the input and then extrapolates. The only reason it can appear to have internal logic is because of the consistent structure of code it was trained on.