You might be right about “overly cautious” but I think doctors being that way is more reputable than gym rats quoting their favorite fitness blog.
We also took insane child mortality numbers and all kinds of defects for granted.
Our mothers still consumed food that has a tendency to contain/grow listeria. Then it was found in several cases where the reason for the death of the child was not known.
Now pregnant women (at least in Europe) are strongly advised to avoid products from raw milk and uncooked meat (hams and salami.
Things change, and not without a reason.
I’m not advocating blind adherence to tradition, I’m simply stating that the Null hypothesis is that soy foods are perfectly fine for health, until actual scientific evidence proves otherwise. All that exists not amounts to nothing more than conjecture, which, given the amount of attention this topic has gotten over the years, one would presume more evidence would be present by now.