"No-one would claim that a modern Harvard Architecture computer with its program stored in ROM isn’t a stored-program computer. So does ENIAC take the prize from Baby as the first electronic stored-program computer?"

I do actually claim that a modern Harvard Architecture computers such as the AVR8 or PIC microcontrollers are not stored-program computers. You can't store a program in them and then execute it. To be fair, some MCUs can change their own Flash so the difference can be subtle - in that case the processor is used either normally or as part of the ISP (in-system programming) circuit at different times.

For very simple stored-program machines the ability to modify running code is needed for it to be Turing complete. In a computer like the Baby, how would you add two arrays? It had no index registers. So you would need to increment the instructions that load and store from the arrays every time you go through the loop. I agree that this isn't an issue on a machine with only 32 words of memory in all, but it is a key idea in theory.

Of course, a Harvard computer can simulate a Von Neumann one (see AVR8 simulating an ARM in order to boot Linux where it does indeed store programs and then run them). In fact, a popular way to implement CISC computers was to build a tiny Harvard machine running a single program in its "microcode ROM" emulating the computer you actually wanted to use.