This is their "small instance" setup.
1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
The usual thought is to use dynamic dns to point to a server you have setup in an EC2 instance, so yeah a third party.
However, this isn't really an all or nothing proposition. You can certainly plug and play different components in to complement your existing setup and still save money / increase reliability.
If you have a solid setup, but are exceeding bandwidth limits you can move static files to S3.
Want better testing rigs, you can make EC2 instances for each and just turn them off when you're done.
EC2 and S3 are great tools, but you still have to do the homework to make them fit your needs.