I wouldn't necessarily agree with renting as long as you can.

You can buy a very cheap server that's much more capable than a cheap VPS server. Here are some decent servers for about $300; you can often find good servers for $150:

http://www.geeks.com/products_sc.asp?Cat=821

Colocation at a good facility costs $60-100.

So for $150-$300 upfront and $60-$100 monthly, you can have a capable server hosted at a good facility at a much cheaper cost than the equivalent "rented" server.

You can use a VPS or something like EC2 as a hot backup, setup replication, and have a very capable setup for not a lot of cost.

This assumes that you have sys admin skills.