Never worked with AWS, but besides that it obviously shouldn't happen - is it really that bad? Couldn't the keys be invalidated/regenerated immediately after you realized they were compromised?
In my case, our CTO was messaging me (either Slack or Hipchat - whatever we were using at the time) within an our or two. Iirc they only managed to accrue a few thousand dollars in charges before we got it under control.