I agree the example could be scarier. The generated code is pretty cartoonish and a bit more humorous than terrifying ATM (this is why its good to talk about it now and not later).

I sincerely hope things will not be so bad, and AI is used to assist and make our softwares more secure, but I think unchecked optimism is also very dangerous.

Instinctually the entire scope of natural language seems like an insanely large attack surface. Securing the entire system will be... extremely difficult. This method is one of nearly infinite strategies that might be available out there.