Colocation isn't that much better either because there are still situations (like corrupt filesystems or malfunctioning power supplies) that would require physical access to the machine.
With an IaaS provider you're free to go live wherever you want and focus on your business and not need to schedule shifts of people within driving distance of the colocation facility on-call.
Ultimately you're paying to reclaim time, which in almost all cases for startups is a worthwhile trade. Outsource everything that isn't your core business.