colocation was expensive and the hardware problems were all mine. i was pretty much tied to my local datacenter because i didn't want to ship a server around (which would be at least a day of downtime). pricing can be hard to compare because of power/space/bandwidth. if the equipment i colocated didn't have IPMI support, it could sometimes take up to a half hour to have a datacenter tech be able to put a remote console online when there were problems. at the end of it, i had a bunch of servers that were worthless on the resale market due to their age.
VPSes were never a serious option for the reasons stated in this article. it's impossible to track down performance problems when a dozen other VPS customers on the same server are taxing the CPUs and disks. i do use one that i pay $10/month for just to run a network monitor for some off-network perspective. they can be useful for single-task servers that don't need a lot of processing power like dns servers.
with dedicated servers, though, you can signup on a website and within a few hours have a complete server with modern CPUs, disks, and lots of memory assembled, tested, and connected to the internet with a remote console waiting for an o/s installation. when hardware goes bad, the server provider has lots of spare parts waiting around to be swapped in for free. and the best part of all, when you're ready to upgrade or move to a different provider, you just cancel the account and let the provider worry about what to do with the old hardware. i have a handful of these on various providers costing between $140-$190 a month for something like a core i5 ~2ghz with 8gb of ram, 2 big sata drives, and 100mbit ethernet with more than enough transfer every month.