Oh yeah, i was pressing all your buttons randomly because none of them were obvious what they do. I was surely engaging with your app very much, while swearing.

But then we actually had a 13% decrease because our test was fundamentally flawed. Also, we don't really understand the statistics behind a/b testing ¯\_(ツ)_/¯

Meanwhile it seems like very few people in the industry actually understand statistics well enough to not shoot themselves in the foot. The number of times I've seen accidental p-hacking is absurd. Having to explain to people why they can't stop running a trial as soon as the numbers look good is like pulling teeth.

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3204791