The main ways it has backfired have been:
1) Once, the removed-thing was used for one team which used it to make one annual report that they didn't realize they were still running. It threw a pretty major wrench in that one report... but they were surprised too, so they quickly modernized it (the old version had known issues), and then that team was able to remove a bunch of dead code because we (the only recipients of its output) could show that X had not been used in a year.
2) And once it revealed bugs in metrics systems / dependency chains... by leading to deleting code that we thought was unused, but was in fact used semi-regularly. An unfortunately-less-than-quick rollback fixed that, and we then went on to find a number of other bugs because moderate chunks of the codebase were receiving no-op metrics emitters for some reason.
Frankly I consider having only ^ those two cases to be luck, but they were both definitely worth it in aggregate. And 2 taught me to verify that surrounding metrics existed / correlated with logs before trusting a lack of signals. I've found multiple other gaps that way :|
3) More than once it has led to not removing code that everyone thought was unused (but we had developed a minor "check first" habit due to past successes), thus needing to maintain some convoluted garbage for almost zero benefit. Bleh, politics.
4) Others blindly applying the habit has led to some careless commits that added it to surprisingly hot loops, adding delay to the point that it caused significantly more timeouts (generally: rapidly filled buffers and paused to flush, then repeat 1000x). The good news is that once they've done this once, essentially every author immediately starts paying attention to their hot loops forever, and we moved more things to batch emissions rather than incremental. Also a net win, it's a pretty cheap lesson.
There was recently a minor annoyance at my work where some firewall rules got deleted because they weren't used during an audit window, and then X months later a deploy got held up because the build server couldn't reach the target servers for that one rarely-changed system.