All execution has to be serial in one way or the other, even if you backtrack you still spent time at T-i at one state, and then go back to a previous state. An LLM could very well act as simply having a tape of the computations it executed and evaluated in the sense that tokens J-K correspond to state at T-i, and the tokens after correspond to the previous state.

In essence, thinking of the computations performed as if logged on an infinitely long tape.

Now, an immutable tape without overwrites is at best a finite automaton, but giving it the ability to replace tokens — which we have and can do — gives us an interpreted tape that — in a very handwavey manner and if you really squeeze your eyes — resembles a Turing machine.

So backtracking could be simulated by an LLM, no?

Is there any “difference” under these conditions up to what an LLM can approximate?