You could argue that basically everything has the potential of being harmful in the long term. Red meat, gas hobs, living in a polluted city, plastic, ... People are exposed to so many substances that we actually know for a fact they are either carcinogenic or bad for our health, and yet most don't do that much about it. If you start cutting out of your life even things that are not demonstrated to be bad for you, you're left with basically nothing. One thing is to have actual evidence that something is harmful, one other is to follow any suspicion that is waved around.