This is not correct. One could say that low levels of asbestos have not statistically been shown to cause harm, but that is quite different from statistically showing evidence of no harm. Harm may very well be occuring, but it is below the sensitivity threshold of our instruments to detect it.
Living near a freeway for instance is substantially more dangerous to your health than occasional incidental exposure to asbestos.
You are breathing asbestos right now. In every breath.
The sensitivity of an instrument to detect harm has no relationship to whether there is some ground truth harm. In many cases, like smoking cigarettes or getting hit on the head repeatedly, harm is incremental and compounding---a single event may not be detectable, but it is almost certainly still harmful.