The Manchester Baby is small, slow and primitive, but it's also fundamentally modern, implementing a proper von Neumann architecture. It had a much more primitive arithmetic unit than the Z3 (capable only of subtraction and negation), but it had an instruction set that would be familiar to a modern programmer.
Zuse didn't know that he had built a Turing-complete computer. He was a practical man and built a practical machine for performing useful calculations. The Manchester Baby was explicitly built as a general-purpose computer. It was in many ways less useful than than the Z3, but it was a far more significant advance. Although Kilburn, Williams and Tootill knew nothing of the work that had been done at Bletchley Park, they were guided by people who did.
IMO Zuse's machines were the big deal, but they were buried under a number of other firsts that weren't that revolutionary, after the war when the Allies were writing history alone for a few years.
While I agree that, conceptually, there is little difference between a program on tape and one in working memory, the latter has a huge speed advantage. More intriguingly, I believe it was thought at the time that the ability of a program to modify itself would be very useful. Turing had already shown that it did not expand what is computable (though I do not know if that was widely appreciated), but even so, when you have very little, and slow, working memory, if it allowed you to reduce the size or speed of your program, that might be a big deal. Even the early COBOL allowed for limited self-modification, in the form of a goto that could be modified during execution to go to different addresses. The problem with self-modifying code turned out to be that it is very hard to reason about and debug.
British machinery like Colossus was still kept a closely guarded secret until the mid 1970s.
Instruction pointer and explicit branching.
The Z3 is still (in a theoretical sense) Turing-complete, even without branching instructions (exponential path construction).
"We can therefore say that, from an abstract theoretical perspective, the computing model of the Z3 is equivalent to the computing model of today's computers. From a practical perspective, and in the way the Z3 was really programmed, it was not equivalent to modern computers."