For SSH, I actually wrote a tool to keep track of all of the hosts (and host groups) that I connect to on a regular basis:

https://github.com/Comcast/ssh-to

I'm responsible for a few hundred servers at this point, and the tool has scaled nicely for connecting to them, and occasional "run this one-off command across a series of servers". (But if it's more than once, you'll seriously want to use Ansible for that...)