Now if you can't afford any downtime, a local server is probably not a good idea, but then neither are many of the cloud alternatives. Also depends on size, of course; one or two local servers is a more reasonable proposition than 35 servers randomly thrown under desks. (Though the "so uh, does anyone remember which room 'thor' is in?" moment used to be a classic startup rite of passage.)
i've done the "drive to fry's, buy a new server, restore from backup" type recovery. it sucks, you don't want to do it, but that's not what i'm talking about. if you're self-hosting at your office your run into all sorts of problems that datacenters solve. the cleaning staff at the DC will never unplug your server. you won't spill beer on it. if the internet goes down, they can fail-over to another connection, your office probably can't. the DC is closer to the backbone, giving your customers less latency. adequate cooling and smoother power means longer life for your hardware. and as you grow it's much easier to grow in a DC than having the server portion of your office gradually take up more and more space.
Most of us have at least ~5Mbps/ 2Mbps (up/down) connections at home that are always on with minimal latency. The gaming service OnLive shows that these connections are adequate for most people's needs. Home/ office connections will only get better, and I think the idea of running one's own server(s) for things like family and/ or small offices will make more sense than some 3rd party corporation's "cloud." PIM, email, IM, pictures, videos etc. simply don't need to be subject to the whims or TOS of a company or outside one's own control.
This is what I do currently, and it certainly is a kludge at the moment, but I believe it can be improved to the point of appliance-usage eventually.
An app that starts getting into the thousands of users will quickly saturate a normal ADSL line (1-2mbps up), and an SDSL/T1 connection is many times more expensive than a remote dedicated server.