Long term web application solution?

It's been a few years but I'm looking to get back into the game and work on a web application. The trouble I'm having is that I need to make it semi-scalable (naturally, as a start-up I don't want to focus on this too greatly) but am also looking for something CHEAP.

It would seem like a cloud solution like Azure, Google App Engine or something else would be optimal but I've also heard it can cost much higher than something traditional.

Nowadays, where should one invest the time and energy when working on a new web application?