If you're drinking whole milk, you're getting a mix of fat and sugar. If you're drinking skim milk, you've removed the fat and kept the sugar. Skim milk's popularity is a result of the low-fat fad (perpetuated by the US government and others) which made everyone scared to eat anything with fat in it. But the fat is actually the good part of milk, so if you don't want the milk fat you're better off picking another drink entirely - like almond milk.