Neumann’s contributions were valuable and notable as they proposed to make machines more flexible than how the ENIAC worked, but these are largely just “applied engineering” improvements not foundational theoretical CS concepts; the Harvard architecture is equally valid and flexible as Neumann’s design, even preceding Neumann’s idea of not differentiating code and data. It just that the simplicity at the time of Neumann machines won out even though today we pay for their penalty in other ways. But without Neumann or Harvard architectures you still end up building machines and likely these ideas would eventually have been proposed and tried by others without any meaningful delay in development. Without Turing’s work it’s possible that computers would have been delayed by decades until someone else figured this stuff out.
Without the transistor computers would just be an odd curio that’s in use in some places and basically the entire field wouldn’t exist (cryptography wouldn’t be a really thing, AI, databases, programming languages etc).
What general purpose architecture should roughly look like was known all the way back to Babbage. Some kind of long list of instructions, with unconditional and conditional branches. Zuse and Atanasoff both got there before WWII.
The big hangup was memory. All early computing struggled with memory limitations, as I mention occasionally. CPUs and instruction decoding came long before enough memory to store programs. Memory was still a million dollars a megabyte in the mid-1970s.
Z3 wasn’t Turing complete and z4 can’t start Turing’s work