But bacteria also become resistant against phages, you would still have to continuously develop new ones.
And particularly if you are introducing a vulnerability instead of directly attacking the bacteria with the phage, there is no evolutionary pressure to become resistant until the phage has already done its job. You can even go further and have it insert genes that confer an advantage in addition to the susceptibility, so that even if some of the bacteria are by chance naturally resistant to the phage they get outcompeted prior to the deployment of the killing agent.