Yes, code executes directly from a literal loop of paper tape, and more specifically, there is only
one loop (the paper tape loop), and
every instructions executes at every iteration.
Yet it can do what any other computer can do (in terms of computation, not I/O), because not every instruction has to actually effectively do something in each iteration, and it is a cornerstone of theoretical computer science that you can transform every program into one represented in an academic language called "WHILE", which is restricted to exactly that principle: It only consists of a single outer WHILE loop, and then inside the loop you have a bunch of conditions.