The biggest selling point was redundancy in the hardware being able to handle CPU/RAM/Disk/PSU failures without taking down the Linux VMs running on it.
Our workload was network bound, not much CPU or Disk IO load.
We ended up using generic Intel PCs (some custom built, some Dell and some IBM Netfinities). We had more down time with the IBM managed Netfinities, then with the custom built rack mount servers.
Back then we also had some Sun UltraSparc E3500 & E6500s, but we found that the Java VM (from Sun!) was much slower on the UltraSparc CPUs then on Intel CPUs.
I find it interesting that we had much better results from generic no brand rack mount servers then most of the expensive "enterprise" class servers.
Note: This was over 10 years ago.
Anyone have more recent experience with servers in a self-managed DC or co-location? It seems most companies/developers now use cloud/vps/hosted servers.