2) "Properly set up" is too ambiguous. Is the whole bucket public? Is the whole bucket private? Are the permissions more fine-grained?
3) Some registrars, like GoDaddy, have been hacked with social engineering. Are you using a registrar that allows password resets over the phone?
4) A password is not enough. You need to enable two-factor authentication on AWS.
Your main points of failure are: AWS, your password manager (if you use one), your OS (if you have malware), and your internet connection.
For AWS, you can enable 2FA and optionally get a physical device that is not your phone. That would help, since your phone is something you carry around a lot and possibly lose.
For your password manager, if you're using a cloud-based manager, you could easily be vulnerable. Who knows if those services are really as secure as they say.
For your OS, you certainly need to have excellent anti-malware software, as well as good security practices (no Java enabled in your browser, up-to-date plugins, no running of unknown/pirated programs or games).
For your internet connection, you should get an HTTP Everywhere-like plugin for your browser. You can't do a whole lot else.
Any executable at all is a potential toehold for an intruder.
Especially remove your compilers. Consider that the Morris worm, as well as lots of malware, enables portability by distributing itself in source code form.
Otherwise it sounds reasonable to keep compilers away from the production servers.
On that note, what are some good schemes for installing self-compiled software to a server? For example, let’s say I run Ubuntu on my servers. On dev host X I have compiled a specific version of Ruby on Rails. Then I want to install that version of Rails to production server Y. I could make a .deb package, but I get a feeling there are other good alternatives as well?
I learnt that lesson the hard way once when I was running python 2.7.3 on my test machine and python 2.7.2 in production.
For a simple program, one that has only an executable, just scp it to /usr/local/bin on your server. I haven't actually done this but it would help to keep a list of all the files you've added, in case you want to remove them later, or reproduce the configuration elsewhere.