Testosterone has been found to improve mood and cause pro-social behavior, not just aggression. I agree that your behavior (and further lifestyle) has an effect on your testosterone levels.
Whether the supposed "male infertility crisis" is actually resulting in any significant real-world consequences is not even close to settled. The average level of sperm is still well within the range of having children without issue, with plenty of "extra room". Worries about a supposed "crisis" are really about extrapolating if it continues to fall linearly for many decades to come, as opposed to simply plateauing at a perfectly functional level.
In other words, it's kind of like saying there's a "handshake crisis" in that, because people are shaking hands with less force, at some point we won't have the strength to shake hands at all. Which is obviously ludicrous (for handshakes).
So whether this is a "crisis" or just a perfectly fine new equilibrium level really depends on whether you think it's caused by environmental poisons accumulating (and so things will get worse), or just a gentler culture that has no need for unnecessarily high testosterone or sperm counts, where everything's still well within "normal" range.
My sperm counts ranged from 70 million to 12 million, so that means I already have more difficulty than I would if my own counts were doubled as at least some of the time my wife and I had sex I was “infertile.”
One more halving and a very large portion of the population will be considered straight up infertile, and the decline in sperm counts shows no sign of showing. Dogs also show the same decline.
Quite frankly I think it’s a far bigger crisis for humanity than global warming.