> Milk has quite a bit of sugar
Sugar content without regard to glycemic index [0] is meaningless. Cow milk contains lactose, which has a very low glycemic index. Most milk replacements use maltose, which has almost double the glycemic load, making them significantly less healthy than actual milk (since they are all basically sugar and canola oil [1]).
[0]: https://onlinelibrary.wiley.com/doi/10.1002/mnfr.201901082