If you dont want to get involved with the sys admin

Heroku for Ruby Apps Google App Engine, Java or Python Windows Azure .net

If you want to maintain the server yourself a VPS is probably the best best Linode/Slicehost

Amazon S3 for static file storage