A minor quibble, though I agree with your point overall: lactose intolerance is not an adaptive response to something harmful in lactose (the way getting sick from alcohol is), it's more of an efficiency hack: since mammals normally only consume lactose as babies, our bodies stop producing the lactase enzyme around the time we stop breast-feeding, so we stop being able to digest this type of sugar, which we would normally never encounter again (or no more than trace amounts).
Some 20k years ago, some of our ancestors started consuming milk even in adulthood, and it turned out a few of them had a mutation that allowed them to continue producing lactase even in adulthood. Since milk is actually a very nutritious substance if you can farm it, these people thrived and passed on their genes, in some areas.