Isn't milk such a small part of most people's diets that it couldn't possibly have much of an effect? Especially for adults. Most people I know just put a bit in coffee and that's about it. I can't remember the last time I drank a glass of milk.
(Note I mean milk the drink, dairy is probably a big part of diets but that's not being studied here)