It's not a problem at all. For example, people do mistakes in calculations all the time but since Math is not faith and/or consensus based nobody flies off the handle and runs a public ostracism campaign on them. You think Bloomberg spent 500M on his campaign so he spent more than if he had given 1M to every citizen of the US? You just look like a fool but there are not going to be 1000 Math professors signing letters to excommunicate you and your bank accounts are not going to be shut down and you still keep your job in New York Times.
If you feel that your science won't stand without harassing heretics, perhaps it is the problem with the soundness of your science and not the folly of the heretics?