Comment by danudey | Hacker News Reader

Comment by danudey | Hacker News Reader

Parent Full thread

My wife and were 'in a relationship' on Facebook within a few months of meeting each other. Then we were 'married'. Even still, for the first four or so years that we had each other on FB, it wouldn't show me her posts. They just weren't in my feed, I'd scroll down so far I was seeing content from months ago, but I'd never see anything she'd posted that week.

Eventually it took me telling FB I didn't want to see posts from 90% of my 'friends', and then going to her feed and 'engaging' with her posts, for it to start showing them to me.

I've never actually used Facebook in any serious way; I'll log in and scroll through my feed once every few months, but I've always hated it. The other day I put my finger on why: it's the mobile equivalent of turning on the TV and flipping through channels looking for something to watch, something you do as an alternative to doing something with your time.

They've probably done testing and found that for some significant percentage of married couples, seeing their spouse's posts caused them to reduce their Facebook usage. So unhappy marriages are ruining it for everyone.

I feel like the general trend of UI simplification and a/b testing everything has companies chasing after anything that will push their general metrics up in the short term, at the expense of slowly disenfranchising a large portion of their users.

I interviewed at Netflix recently for a position on their UI team, and I questioned the interviewer about a lot of annoying aspects of the Netflix interface. He was aware of every complaint, and his explanation for every one of them was he wishes he could change it but the a/b testing shows they get higher engagement with it the way it is. So, for example, they show you movies you have already said you don't like, because enough people give a thumbs down and then later watch the movie. In the short term very few people are going to cancel Netflix because of this, but enough of these decisions in succession is erosive to customer goodwill.

Personally, these kind of decisions wouldn't bother me that much if apps and websites made it easy to customize your experience. Facebook makes it difficult by opaquely shaping your newsfeed and giving you what seems like hundreds of options across a half dozen settings pages. Netflix makes it difficult by just giving you practically no options, but there's enough demand that's there a tiny industry in just making websites and plugins to make Netflix usable.

yes, a/b testing optimizes for an average user that dosent exist, and everyone is unhappy with.

Reminds me of the old tale of the man who drowned in a lake with an average depth of 3 feet.

A/B doesn’t design for an average user; it can be all, a tiny niche, or something in between depending on what you’re measuring. For example, DAU and engagement will be affected by drastically different distributions of users.

I guess A/B testing will tell you if the horse will become faster if you change from iron shoes to carbon fiber ones.

A/B testing won’t ever get you a Tesla from a horse.

It will also tell you that pumping horses full of steroids will make them go faster. Tech's absurd obsession with metrics and statistics will be their undoing.

If you do enough binary switching, you eventually will get a Tesla. Assuming a Tesla is optimal. Let’s test to find out.

How do you think the eyeball arose from unicellular eukaryotes?

local maxima trap hill climbing style optimizing, many (most?) times improvements require jumping large sub optimal chasms that can never be crossed by gradual improvement.

You won't get out of local minima/maxima with hill climbing though.

Eyeballs are nowhere near optimal either.

I don't think so. A/B testing is like gradient descent which is a greedy algorithm. You move in the direction that locally looks best. Evolution on the other hand allows for suboptimal species to persist for enough time to let them develop their advantage. (In the language of optimization evolution allows you to go past the local optimal and reach global optima by allowing you to move in non-optimal direction -- as long as the move is not catastrophic.)

Nope. A/B testing depends on what you choose to mutate. The problem is that we humans intentionally change the things. Nature randomly changes the things.

You’re not going to get to a global optimum driven by human choice of what to test (only local optima at best) unless the human setting up the tests is some sort of sage.

Do you Nature will eventually hit on a global optimum? If so, that belief is opposed by what biologists know about developmental and evolutionary pathways.

For a trivial and well-known example, there is now no developmental or evolutionary pathway that will lead to a vertebrate eye where the nerves run behind the retina. As a result every last vertebrate has a blind spot where all of the nerves dive through the retina. This makes our eyes less efficient because the nerves block out light.

We're therefore stuck with the first design of the vertebrate eye and can't change it. There is no pathway to the more sensible design of the mollusk eye.

This is but one of many examples. For another one, relative developmental timing of growth is fixed across vertebrates. For example the "hand" grows before the "arm". The result is that pterodactyls, whose wings are entirely hands, could fly from birth. But birds (whose wings are arms) and bats (a mix of the two) can't fly from birth. No matter how desirable an evolutionary trait that may be.

Now I wonder, has someone ran A/B tests but instead of using hill climbing for optimizing used simulated annealing?

> Do you think Nature will eventually hit on a global optimum?

No. At least not in the way described. Optimal as considered by whom? Us? We look at something and say this would function better than that, but is something that is optimal perfect? Basic microeconomics shows that perfection is sub-optimal due to the law of diminishing marginal return. Perfection is wasteful and unsustainable. What do we know to be optimal? From what I know, “adequate enough to reproduce” may not be perfect, but it might be brushing up against optimal in its “good enough and no more” / lagom nature. Mollusks have mollusk eyes. Can we know that humans would be better off with mollusk eyes without rewriting our evolutionary lineage for us to be more like mollusks in other imperfect ways?

Says every product person I've ever worked with who didn't want to actually be data driven.

There are two kinds of product people: those who can't design and those who don't understand statistics.

I understand the fear, though - when you're data driven, you can actually numerically measure the contribution of every feature that's tested, and in a lot of cases figure out the exact impact on revenue. In a culture where everything is tested, it's a very small step from there to stack ranking your product designers based mostly on how the features they designed did.

I'm not necessarily against that as a valid way to measure job performance if it's done intelligently (for one, realizing that there's a lot of blind luck and variance, and it takes time to smooth out) - I mean, if you're in sales and you're not booking any sales, you don't get to hide from that. But it's also really easy to get ranking and evaluation schemes wrong, so I understand why people would be nervous about it and prefer soft-skill-based evaluations instead.

valid point. However I also often see "politics optimized sales" where the current pet project of whoever has the most swing goes up front even if it's a terrible product and death to revenue.

A/B testing won’t ever get you a Tesla from a horse.

Right. But if you were already selling Teslas, and along came a smooth-talking product designer with a dream to "improve" it by building a horse instead, A/B testing would make sure that change never saw the light of day.

A/B testing is not a way to get out of having to come up with good ideas yourself. It's a way to validate that your ideas are any good in the first place before betting the whole company on them.

A/B testing is most critical when evaluating big changes to a product, because those are the changes most likely to completely blow up the business. Otherwise it's left up to the opinion of the highest paid person in the room, and people are notoriously bad at guessing how customers will respond to change.

Indeed. When I was advocating for A/B testing at the last company I worked for, I tried hard to teach people that every test should have a sane hypothesis behind it based on some kind of usability theory. Not just “let’s try changing the header font color to red and see what happens!”

I worked for a company with an obsessed AB culture. Biggest hole I found. Clicks != satisfaction. Time spent on site also != satisfaction.

There’s something very powerful just talking to the user, feeling their pain and working with them to fix it until they say “this is awesome”.

Did you ask what their definition of engagement is, and why it's the primary measure?

> In the short term very few people are going to cancel Netflix because of this, but enough of these decisions in succession is erosive to customer goodwill.

The “auto preview” is so unpleasant that I’ve essentially stopped using Netflix entirely. I’ve no doubt that their testing shows it increases engagement, but I find it so jarring and abrasive that I rarely open the app any more. I’m sure they still have content I want to watch, but I can’t stand the experience of trying to find it.

I would love to have more insight into their A/B engagement metrics. I’m sure they capture if immediate engagement goes up when a feature is enabled, and probably the same over time periods of days or weeks. Do they also capture if total app usage declines 75% over 6 months after a feature is enabled?

That’s funny - I Like the auto preview, in fact I would like them to go further and advertise their content to me, as live TV does. When I watch Netflix at the end of a hard day I’m in lazy consumer mode, I want content to be suggested to me. I hate wading through static dull pictures which give me very few clues as to the content they represent. Netflix are unstoppable - they offer incredible value compared to extortionate SKY, i cancelled their £120 per month package 6 months ago and I have never regretted it.

I'm listening to music or half-watching twitch whilst scrolling through Netflix to decide what I'll watch with supper. It's incredibly frustrating to have them autoplay video content, to the extent that I've started going back to browsing torrent sites instead.

It doesn't help that their website is a laggy piece of shit with miserable discover-ability. But, you know. Ugh.

Surely the video is muted by default, why is that a problem?

It is not muted by default. Nor is there any way to mute short of muting your TV.

The website doesn't autoplay, but per Ntrails, it's "a laggy piece of shit with miserable discover-ability". I don't actually use the website so don't have an opinion there.

I also like auto preview. But I would never want them to start advertising their shows to me (like Amazon Prime does), that's going too far.

I agree. You can't even leave Netflix running on a device and leave the room because it will just start playing something without any actual user input! And it appears none of the apps let you turn this behavior off.

Netflix already seems to have no problem getting people to watch hours and hours of shows as a result of having high quality content. Why do they need to perform cheap tricks like auto-preview to bump up user stats, as others have pointed out, at the expense of user happiness/preference?

> The “auto preview” is so unpleasant that I’ve essentially stopped using Netflix entirely.

If you find a lower powered device such as the previous generation of Amazon Fire TV, it does not auto-preview or auto-run trailers at the top of the screen - instead you just get a static image of the show.

Now if only I could get it to do that on my PS4 :(

I'm definitely not planning to buy a dedicated crappy device for Netflix, though. :/ I want fewer devices. I've only still got an Xbox One for Blu-Ray/DVD and HBO Now, which inexplicably isn't available for my TV.

> he wishes he could change it but the a/b testing shows they get higher engagement with it the way it is

The upshot of this is that if Netflix or Facebook _do_ see a slow drift of users away from the platform, they are institutionally incapable of understanding why. Engagement is not the same thing as satisfaction, which is far harder to measure.

(I'm also now wondering how static the results of A/B testing really are - if an idea fails AB even once does the company abandon it forever?)

But engagement is what advertisers pay for, isn't it ? Clicks ! Clicks ! Clicks !

So I'm pretty sure that they are institutionally incapable of understanding why for a very good reason.

Geeze, I had the EXACT same issue. I almost NEVER see stuff from my wife (even after I marked her stuff as something I want to see). I always see friends that I never interact with and their events that they're going to or all of the meme pages my friends like. But my wife's stuff? Not very much.

I hate using Facebook. The only reason I haven't removed my account is because it's the only way to keep up with certain friends and family. I feel like I am forced to use it simply because there isn't anything better for them to broadcast on.

> The only reason I haven't removed my account is because it's the only way to keep up with certain friends and family.

I'd suggest considering whether this is really true.

That's what the follow feature is for. You can get notified on ALL activity of a friend of you are willing to click two.

All of my friends are set to follow. Seems like a default option.

Yes, but there's a "see first" option and you can turn on notifications on activities:

https://imgur.com/a/xZTQk

You'd think Facebook's algorithms would prioritize your spouse, but you could use friend lists to make it do what you seem to want. You can add people to your "close friends" list and you'll get notified whenever they post. Also tends to put their posts in your feed more from what I can tell (based on the minimal amount that I use FB these days).

https://www.facebook.com/help/598069963644156

Facebook's weighting there makes sense to me. Why do you need to see your misso's posts? You see her all the time.

The posts I actually want to see on Facebook are from my friends that are far away. For me though, FB weights posts that are geographically closer to you higher. When I moved to Australia, I discovered through Facebook that I had 3 or 4 friends from high school here in Melbourne, purely because their posts started showing up on my newsfeed.

Can completely relate. The other annoying thing I have found is that FB will show posts that are outdated. For example, prior to replying here, I scrolled through my feed and over half the posts were from yesterday or two days ago. Even more frustrating is when those posts are from media outlets and contain outdated news or other articles of interest. I know the SOTU was last night, I don't need several posts from various media outlets trying to tell me where to watch. I know the problem is I don't engage often enough or with the content I want to see, but it would be nice if I could have more control over what content I see in my timeline without having to engage.

Reply on news.ycombinator.com