Besides my database servers, all of my machines are ephemeral[1]. I can lose machines and spin up new ones automatically (and therefore scale elastically).

EBS adds an unnecessary additional point of failure that I want to avoid at all costs.

Having run production systems in EC2 since 2010, I've survived numerous EBS outages as other services have crashed and burned by sticking to this philosophy.

[1]: I don't use EBS for my database servers either. I use a replicated DB that can lose multiple machines simultaneously without loss of data (and is backed up to S3 in case of a catastrophic event).

There's no question that what you sacrifice in performance with EBS, you get back in easy durability. But if you design your persistence layer with replicated instances, you can have the performance of instance storage. You'll need to make backups yourself, so it is a bit of extra work over making EBS snapshots, but IMO the performance gains are well worth it.

Ephemeral give you extremely high IOPS for no additional cost. It's perfect if you don't need a ton of disk space. A great example is Graphite. Very high IOPS requirement but (usually) a low disk space requirement (assuming you configure relatively low retention) The c3.2xlarge boxes are great for that.

The equivalent on EBS on the other hand will cost you an extra $250/mo or so in addition to your server cost. And you will cap out at 4000 IOPS.

EBS disks are more durable, sure. When you can reach them. You should be designing for failure, and EBS fails not infrequently relative to the rest of AWS. The problem is, when it fails, your instances go with it.

Individual machines' durability is almost irrelevant. Your service's durability is very relevant. This is the most critical consideration in almost all cases in the cloud. EBS makes service durability much harder.