IMO, this is seeing everything as a nail when you're a hammer.

AWS provides extreme flexibility. This really only becomes valuable once you hit a scale that you can optimize your infrastructure around the shape of your needs.

* Need some heavy workload to run in the most cost effective data center at the most cost effective time. Yep, AWS is great for that.

* Need a specialized GPU set. Yep, AWS is great for that.

* Need some crazy storage or memory setup. Yep, AWS is great for that.

For everything else, you should really consider using a PaaS with a good ol' Docker container.