Alcohol, plastics (which leech from containers into the liquid they contain), pesticides (on fruits, vegetables)—there are a host of environmental factors that reduce our testosterone levels.
And as for anxiety, I think our modern urban lifestyle far increases our anxiety levels compared to our ancestors.
The human body was not meant for the world we live in!