You gotta receive a ton of traffic for Nginx to stop serving static files. I'm pretty sure most of the static websites today could survive on a lite Nginx vps with minimal tuning.

Also, i rather have my server go down than to receive a larger bill from aws, but that all depends on your use case obviously