The ones that can digest it, do it usually less well than they think they do. Mild chronic diarrhea is prevalent is western countries, and a degree of dairy intolerance could very well be one of the factors.
It gets worse for most people with age, as elderly people can become lactose intolerant, fully or only slightly. This does not happen overnight, its a progressive thing, many people gradually stop being able to properly digest dairy.