I think that something like Stack's Blackbox is the best idea. This ansible-based setup also explains the concepts pretty well: http://ansiblecookbook.com/html/en.html#how-do-i-store-priva...