Yes... A real Atari does the same thing! That's what the processor and the other chips produce when the machine is started with no ROM to be executed.
I tried to implement the emulation at the hardware component level as accurately as I could. It happened spontaneously!