In addition to the lost time, you'll have a hard time racking the slide from the adrenaline dump. Also, the bad guys always have first-movers advantage and you're always playing catch up.
After watching thousands of violent encounters on John Correia's "Active Self Protection" channel, I agree that round-in-the-chamber is absolutely necessary if you're carrying for self protection.