There has been a significant population level decrease in testosterone in men. It's at around 1.2% per year when adjusted by age.[0] Similar effects have been found in other countries too.

I think this is what he means when he says physiological.

[0] https://academic.oup.com/jcem/article/92/1/196/2598434