This made me think of the feature flipping that Facebook Engineering uses - details here: http://news.ycombinator.com/item?id=2594083