A neural network doesn't understand anything but sure it could predict the program output sequence based on the input sequence.
Would there be any useful use case to such a technology?
The counterexample is for a transformer with context width of N tokens, to try to model "def f(x) { ... }; <at least N more tokens here>; def g(x) { y = f(x) ...};". In this case, in trying to evaluate "g(4)", f appears as the literal token "f" and nothing more. To be able to learn to evaluate g, a model would need to be able to do (at least) distant coreference resolution.
I've had the displeasure of trying to explain classical learning hardness results to someone who kept repeating back to me "but RNNs are Turing complete!" If anything, the expressive power of the model you're trying to train pushes back against your efforts to train it to do something useful.
For example while toying around with it, I gave it a javascript function that referenced an undefined callback, and GPT-2 gave me back a function, with almost correct syntax and mostly random logic, but with the name and call signature of the callback.