Maybe because the background radiation is just one of many, many contributing causes of cancer?

For instance, airline pilots are exposed to more radiation than the average worker. But they also have a low percentage of smokers, and in order to keep their medical license they need to be in fairly good health. So, apart from skin cancer, they have lower than average rates of cancer compared to the general population:

https://www.alpa.org/portals/alpa/magazine/2001/March2001_He...