That's true, but then you start running into weird compatibility issues, where a specific game won't work because an Xbox Live component is not installed or DirectX SDK is not at the right version (and you can't install the right version because it only works on the consumer OS).

Server 2012 (or was it 2008?) was absolutely terrible in that was absolutely Windows 8-ified with a forced full-screen metro start menu "experience," but that was fortunately rectified in the next release (Server 2016, I'm pretty sure).