It's a few years since I've used AWS, but is deleting stuff that hard? I seem to remember it was a matter of deleting a cloud formation stack and maybe cleaning up a few things that you'd explicitly ask to be preserved (usually storage things like S3 buckets and DBs).