...As data you can access as a filesystem. S3 is great, but pretending it's a filesystem is going to get you awful performance.

As I said, our data is not lost, we have snapshots and backups, it's sitting right there on the mount, we're just not getting any sort of acceptable throughput. New instances does not fix the problem.

Ironically, we were looking into having S3 as the backend for our data, for scalability/redundancy purposes, but this pretty much puts a stop to that.