No, you'd need to be chugging multiple jugs of soy milk a day for that to (maybe) matter. Better would be to lift, eat better, sleep, etc.
You’ll be surprised. Some vegan friendly foods which are delicious are packed with more than 1x your daily value in a single serving. So if you eat two a day for instance the levels of estrogen you’re pumping in are quite significant and very very easy to do.
It seems that at least in rabbits[1] a diet consisting of nothing but soybeans had no effect on fertility.