You’ll be surprised. Some vegan friendly foods which are delicious are packed with more than 1x your daily value in a single serving. So if you eat two a day for instance the levels of estrogen you’re pumping in are quite significant and very very easy to do.
It seems that at least in rabbits[1] a diet consisting of nothing but soybeans had no effect on fertility.