That coal waste is also radioactive. Natural, but still radioactive.
To me, it's like someone asking where me to choose which part of my house to infect with wood rot:
1. A single scrap of wood, extremely badly affected, stored in a sealed box under the stairs
2. My entire roof mildly affected by rot.
I'd take the box under the stairs.
Asbestos is naturally occurring and incredibly dangerous. "natural" has nothing to do with risk to human life.
Not sure who you think you're disagreeing with? Nobody said otherwise if you read the thread.