Ask HN: What's the most absurd A/B test result you've seen? | Hacker News Reader

Ask HN: What's the most absurd A/B test result you've seen?

24 pointssAbakumoff19 comments

I don't know if I have a good answer for this specifically, but as a consumer, I have a little rant. You know, Amazon must A/B test the shit out of their homepage... but if I buy a shower curtain, my Amazon home page is nothing but shower curtains for weeks. How many shower curtains do they think I need? This works with other once-in-a-blue-moon items, too: plungers, watches, windshield wipers, phone cases.

What they should do is say "Hey, this guy bought a shower curtain, which means A) he has a bathroom, and B) either moved in or is redecorating... and that means he probably wants a bath mat, a toothbrush holder, a plunger, a towel rack..."

But nope... shower curtains for days. And you know that somehow this works out great for Amazon, or else they'd change it.

"Turns out the last shower curtain you bought from us isn't working out so great, is it? Well, check out all these top-ranked shower curtains that your search didn't return!"

Brand recognition by repetition? Even if the brands themselves aren't buying ads, maybe there's a bigger chance that in the future you'll feel more comfortable buying that item again. So it's better for Amazon too.

Potentially!

I suspect "shower curtains for days" A/B tests better than anything else they have to test it against. Not that it's the absolute best thing, just the best thing they can automatically suggest. A bath mat that's a color/pattern match for that shower curtain, for example, might convert really well. But one that's not will convert poorly.

I am surprised, as you hint, that they can't tell "one time" versus recurring. Where it should pitch me AA batteries I bought 6 months ago versus the more recent shower curtains, since I will likely buy the former again.

Like I said, I'm positive they A/B test the crap out of it, so I'm sure it brings in more conversions... but that is sooo counter-intuitive, I'm not sure what the rationale is.

Maybe just showing you more shower curtains is enough to remind you to shop for those other bathroom items?

It's possible. Like I said, I'm sure they A/B tested the shit out of this... I just don't know why this keeps happening. Your hypothesis is as good as any right now!

Perhaps B was.much,much worse.

That's my assumption! I think A is a bizarre solution, but I absolutely think they've tested the crap out of this and that the way it works is the one that makes the most conversions for them. Still... crazy!

Relatedly, it would be good for their spam system to be aware of purchases. Case in point: I recently purchased (in pre-order) a movie Bluray on Amazon. Only a few days later, I received spam telling me that very same movie was now available for rent on Amazon video (after the release date and after I actually received the disk).

This happens to me all the time, as well. I read a lot, and I buy a book on my Kindle and two days later get a promo email with the same book in it. I imagine their emails are just less sophisticated and target tens of thousands of customers per batch, rather than sending individuals perfectly tailored emails... Still, you're right. It's annoying.

https://twitter.com/actioncookbook/status/834439563032555521

   ME: [views product online] Hmm. Nah.
   [changes website]
   PRODUCT: we meet again
   ME: Sorry, no [changes again]
   PRODUCT: bitch this ain't over
   ME: FINE. FINE. I will buy this rug. Just leave me at peace.
   REST OF INTERNET: this dude loves rugs, let's get him, boys

Had the same issue when I bought an umbrella recently. My Amazon homepage is nothing but umbrellas now. I know I live in the UK but come on, this is overkill, I don't need that many umbrellas.

This article covers it well: https://theawl.com/a-complete-taxonomy-of-internet-chum-de0b...

It's about the ads you usually find at the side and bottom of dodgy sites. An exact quote from the site:

"Like everything else on the internet, traffic flowing through chumboxes must be tracked in order for everyone to be paid. Each box in the grid’s performance can be tracked both individually and in context of its neighbors. This allows them to be highly optimized; some chum is clearly better than others. As a byproduct of this optimization, an aesthetic has arisen. An effective chumbox clearly plays on reflex and the subconscious. The chumbox aesthetic broadcasts our most basic, libidinal, electrical desires back at us. And gets us to click."

So you get all kinds of disgusting, disturbing and yet fascinating things that entice you to click. Clicking the most disturbing thing encourages more content like that.

I knew a background check website that tested two funnels. One gave the free info instantly and then upsold to the paid plan, the other gave that same info after an 11 minute B.S. process. The version that took an artificial 11 minutes led to literally 10x conversions to the paid info.

I think this makes sense. If they can get it instantly without pain there is no incentive to upgrade. Having to wait is extremely annoying.

A dating website I used to use does something similar. It makes guys wait 10 minutes between each message while girls can send all they want. I can imagine this works really well because even I considered purchasing it, and I never purchase such stuff.

You find a nice lady?

Mostly when significantly different pages make little to no difference on conversion. This has made me a little jaded about sweating over the colour of a button, or wording a statement type thing. Yes, a site needs good architecture and design. But overall its other things than design look/language that become the real needle movers for digital performance, but people often focus here as its the fun stuff.

Reply on news.ycombinator.com