I think that's what happened to me. I can consume milk products except for liquid milk which will make me sick. (I haven't had much trouble when I use it in cooking though.) This is after going on soy for a year, where before that I could drink milk in the truly heroic quantities being discussed elsewhere.
I'm pretty sure if I tried to copy ImprovedSilence's habits now, I would die. I know I would wish for it :-)