Microsoft expects me to buy a separate, full Windows license for each virtual machine I create, for each host machine I run it on.

Yes, that's what the license says.

In practice, what corporate VM users do is get MSDN licenses, which are assigned to the developer. I have one. It's basically a license to not worry about licensing: I can run as many copies of whatever I like, so long as it's "for testing and development purposes". I think there's a different volume licensing programme available if, god help you, you wanted to run a production datacentre on Windows.

Edit: this is obviously a big reason why Windows is never going anywhere in the cloud space outside of Azure.