Server 2003 was my daily driver on a laptop for a few years. Most XP software ran fine but it had none of the XP visual cruft. Until Windows 7 it was peak Windows for me.
Driver compatibility was terrible, as you might expect, but it could support a vast amount of memory (relative to XP). I used the base O/S only to run VMware Workstation, and then ran small instances of XP, each dedicated to a task like audio, video, etc, on top of that. Along with FreeBSD and various flavors of Linux.
That was my daily driver until late 2010. Solid as a rock.
I had a Customer who usd it to run Unigraphics CAD. The Dell Precision machines they had with factory-loaded XP64 were fine. Trying to get their random other machines to run it w/ working audio, video, and chipset drivers was a pain.
I never ran it as a desktop OS myself. I installed plenty of the server version it was based on (W2K3 x64) but those boxes always had standard VGA drivers, no audio, and sometimes a RAID controller driver, but that was usually it.