If you're running in "the cloud", you should _always_ be able to destroy your instances and have them auto-rebuild from configuration management and a persistent object storage system (git repo + S3 or Google Nearline).
I just don't even know where to begin. Is this something I could stand up locally and push to like s3, spin up a new server and install 1 thing and have it pull the configs and software installs down?
Any suggestions?
My opinion is ansible. Basically python + yml, you list out steps to set up your server. Takes a few minutes longer round 1, but you can remake the server in 60 seconds in 6 months. Lots of ansible tutorials around, try it out..
user data lives in redundant databases / storage service
server setup lives in a repo via bash scripts or ansible etc (and bake that into an ami or container)
setup your launch-configuration for your servers to setup all of the above and you can destroy and spin up servers on a whim
Yes. You could use shell scripts, Ansible, Salt, Puppet, whatever. I'm fond of Ansible or shell scripts, depending on complexity.
Start here: http://www.opsschool.org/en/latest/config_management.html