Things are not so cut and dry. For instance, did you know that a diet which includes fermented dairy populates the gut biome with bacteria that will break it down for you[1][2]? And that the placebo effect plays a role in symptoms of lactose intolerance[3]? The idea that milk wasn't drunk in East Asia in antiquity is a misconception as well. It was artificially dropped in Ancient China while assimilating pastoral societies as part of a larger scale dietary reform[4].

Additionally, cow milk represents a single dairy product amongst a plethora of ones. Everything from Horse milk to Goat's milk is consumed globally, and much dairy is consumed as further processed food like fermented milk, yogurt, cheeses, etc. which generally aren't considered by Americans when talking about dairy, for whatever reason, even when supposedly attempting to "unplug" from the dairy industry's propaganda.

[1] - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4066025/

[2] - https://pubmed.ncbi.nlm.nih.gov/15277142/

[3] - https://gut.bmj.com/content/41/5/632

[4] - https://www.asianstudies.org/publications/eaa/archives/china...