I work in this field. Author makes great points. Here are some additional thoughts: * Don't use bad optimization to discredit all optimization * Incremental A/B testing assumes that all accumulated features are independent with no interactions, which is almost assuredly a poor assumption as a website grows and space becomes limited * A/B testing is the only reliable tool to test hypotheses; causal inference is great but not a substitute * Incremental A/B testing (small changes) should be periodically coupled with mutational A/B testing (large changes) to tunnel from local optima to global optima