#2 is totaly relevant to our Macro Recorder app.

Macro Recorder is really popular and most of the visitors are already "pre-sold". They don't care about the landing page.

We used to run tests on that page, because it has 1k daily visitors, we thought that we could test ideas more quickly. That was before we realised, that we can't rely on the results.

We even used false conclusions, based on those results, to make changes to landing pages for other products. Perhaps the biggest mistake we've made.