The glycemic index of most milk substitutes (oat,soy,almond) are equivalent or greater than a can of coke. Milk substitutes also tend to be really really high in vegetable oils that seem to at least have a troubling correlation with chronic disease.
The metrics used for the environmental impact of intensive animal husbandry tend too skew towards the worst case (obese cows on a grain fed diet in a feedlot). There are plenty of dairy farms raising cows with a more sustainable grass-based diet that doesn't release nearly as much methane (Think how gassey you'd be if the only thing you ate was ice-cream?)
I like supporting my local dairy farmers, more than I enjoy supporting some multi-national milk-substitute beverage company with a supply chain spanning the globe.