AWS is incredibly convenient, but it's priced at a premium. The main cost liability to building on AWS is the cost of bandwidth is insane, 5X the cost of DigitalOcean or Linode, 10X the cost of co-location.

Of course this depends on the type of startup you are building, a video startup is gonna be crazy expensive on AWS, Non-media heavy SAAS it may not matter.

Of course you can always build on AWS and place a caching layer in front of your bandwidth heavy media using DigitalOcean boxes to save big.