Even if they destroy them within say a thousand years, then at any given time there would be only few civilizations at the right age to be broadcasting. And not very loud at that.
We listen only to a miniscule portion of the Galaxy. IMO, Fermi's question shouldn't be "Why can't we hear the few civs that are close to our level".
It should be "How come we don't detect Kardashev >2 civs collonizing most of the galaxy?" Why can't we detect a single Dyson Sphere?
(I don't consider dark forest explanation to be probable though.)