HNHacker News
TopNewBestAskShowJobs

ernopp

327 karma · joined January 7, 2014

submissionscomments
ernopp··on Google’s constant product shutdowns are damaging its brand
> then suddenly get an email that Google is discontinuing the product.

Google is unlikely to discontinue GCP

ernopp··on Microsoft is now worth more than Alphabet
IMO: Outlook on iOS is a better email client than Apple Mail.
ernopp··on Microsoft is now worth more than Alphabet
Gmail and Android can be considered moonshots, completely separate bets from the cash cow search product. They have both been enormously successful and been the starting point for new revenue lines (the whole GSuite for businesses, the Play Store)

Next moonshot which looks extremely promising is Waymo

ernopp··on Use of WhatsApp in NHS 'widespread', say doctors
And the NHS is the world's largest purchasers of fax machines in the world in 2017 http://www.nationalhealthexecutive.com/Health-Care-News/digi...
ernopp··on Uber Is About To Launch An API
can't wait for uber for uber
ernopp··on What was your best passive income in 2014?
just from ads?
ernopp··on Sense – Wake up when it's right for you
the kickstarter is surely to find & take orders from an audience for the soft launch
ernopp··on 2 Step Verification as a Service: 2StepsForApps
great service, makes life easier..
ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Full disclosure: I work for Qubit who published this white paper.

Maybe the confusion here is in tests which have a "true" effect and an "observed" effect. If an experiment has a true effect, then you have some chance to observe it, which is the power.

But false positives have by definition already been observed as winners (that's what false positives are), so there's no need to apply the factor of 0.8 to them.

ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Just to let you know it's been updated.
ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Well it depends on your conversion rate. But assuming it's ~few percent, then yes it will be hard to measure anything other than very very large effects in conversion rate unless you're willing to wait quite a long time.

You can measure more upper-funnel things though like I said in the GP, which can be very helpful, especially in combination with qualitative feedback, although this depends on what exactly your business is...

ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Full disclosure: I work for Qubit who published this white paper.

This kind of "historical testing" (I think people often call it sequential testing?) can be pretty dangerous even for large effects. For example Christmas might be a really good time to change the colour of all the buttons on your site and see a 50% increase in sales.

ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Full disclosure: I work for Qubit who published this white paper.

Just wanted to add that if you have less than a million users you can A/B test for upper funnel goals, effectively measuring if changes improve engagement. Obviously then you have the problem of working out if the engagement translates into more sales but perhaps you're willing to wait longer to find out if a test that improves engagement leads to more revenue in the long run.

ernopp··on Most Winning A/B Test Results are Illusory [pdf]
Full disclosure: I work for Qubit who published this white paper.

I see a lot of this kind of testing going on in the industry and it's frustrating. A/B testing can be a massive tool for your business if it's done right but obviously if you only wait for 2-3 conversions, you're not learning much... "Good" to hear that other people feel the same way!

← PreviousPage 2 of 2