The source[0] below seems to contradict what you're saying about sustainability, at least with regard to land use - which seems (to me) to be the most pertinent issue in that aspect.
"Livestock takes up nearly 80% of global agricultural land, yet produces less than 20% of the world’s supply of calories (as shown in the visualization). This means that what we eat is more important than how much we eat in determining the amount of land required to produce our food."
I know you aren't supporting traditional livestock (red meat), but the breakdown of the numbers from the UN Food and Agriculture Organization shows that plant-based food sources already account for 83% of the global calorie supply using only 23% of "Agriculture land". Based on that land use/calories ratio, we could increase to 100% by taking only ~4.7% of the total agriculture land from the livestock portion. If we assume that we definitely need all the protein listed, that would have to be increased to ~6.2%.
Obviously there would be other issues, but if we could return most of the remaining portion of the livestock land to its natural state (even scrub land can benefit from reducing livestock grazing), that seems like an enormous gain for sustainability.
Of course, this doesn't diminish your initial point about eating fish in a sustainable manner. How we could enforce the sustainable aspect is a another matter, given the number of news stories of issues with fish farms and continued over-fishing in protected areas, but that's beside the point. I just wanted to flag other potential gains from more vegetarian diets.
[0] https://ourworldindata.org/agricultural-land-by-global-diets