I doubt a single atom of lead will be unsafe. Or two. There is definitely a level at which the intake of lead will not lead to any measurable effects during the lifetime of a human.

However, the question is about how much it matters compared to global intake.

We get lead from many sources, some of them hard to avoid while still living a normal life. "Safe" doses are doses that represent only a small fraction of that unavoidable part. The standards may change as the unavoidable become avoidable.

For radiation, the baseline is clear, that's background level, I don't know the equivalent for lead but it's clear that we can't keep away every single lead atom.