You can calculate the minimum sample size needed to come up with that threshold rather easily. See: http://vuurr.com/split-testing-determine-sample-size/
If you're going the classical statistics route the entire point is that you need to determine your sample size before you peek at the data. In that post you would need to replace E with a threshold of difference that you care about, then calculate n before you start the test and not look at the results until you had reached n observations.