Of course, installing everything on more expensive EC2/rackspace machine / mysql service would take half of a day, ....
Using heroku would be way more easier, but heroku is not so cool and hippie as Google.
GAE, through Java, can run a multitude of languages, basically everything that runs on the JVM. Not every language is a good choice for GAE, but if you want, you can.
You’re right about installing many things on EC2/Rackspace is becoming easier every day. But tuning the system to get the most out of the infrastructure requires advanced knowledge in all layers of the stack. On appengine, you just have to write your app, you don't worry about the infrastructure. It’s a fairly good thing for many people.
With the generous free quotas, you don't have to pay a cent until your project gets some traction, which is not the case with EC2 for instance, because they charge you by the hour and not by the amount of resources used.
GAE is not perfect, but it enables small teams to rapidly launch prototype and iterate until the project takes off. Whether you have 10 users per day or 10M, you’re running on the same stack, built by some of the brightest engineers at Google.
We spent three years working on DotSpots on the EC2 platform. While you can certainly boot up an instance of your application and get it running, there's still a lot more infrastructure work required to keep it going, scale beyond a single webserver, etc. On top of that, you've got to keep something like Pagerduty running to make sure that your instances aren't wedged or your MySQL box isn't running close to its storage limit.
The big benefit for me was being able to chose a language I'm strong in and just deploy something that automatically scales. This is particularly important during the early phase of the project where you are a lone coder doing all of the work.
Of course Heroku can be easier if you're using Ruby/RoR, but again using Ruby means Heroku is the right tool. Heroku is useless for someone that wants to use Java or Python to build their site.
At the end of the day a real entrepreneur doesn't care what's cool or hip and instead one focuses on the end product and how he can deliver it.