The body does finish them off in a way. They wall off areas of the lung for example, making "tubercles", that contain tissue and latent bacteria.

Future ill health allow these to reactivate.