My understanding is that almond milk is not good, but is far better than cow's milk. Oat milk supposedly has the least impact. See the chart in the article I linked in up-thread.
It has ~20% of the protein and fat content of regular whole milk, so at the very minimum it has to be 5x environmentally friendly.
In fact it is so nutritionally empty, that it is more fitting to compare it to slightly sugared water, and see if it beats that on environmental record.
I think more people drink milk purely out of habit/pleasure.