A lot of people here agree with you about Meta's faults, even me. But that doesn't mean we can't appreciate the good they've done. Even if it isn't much and even if it is vastly outweighed by the bad. We should still use positive reinforcement to pressure companies to do the right thing.
But you are totally right. We must celebrate with he good in others and life or our critical observations lose all merit.
What's frustrating is that they currently have a near-monopoly on friend-to-friend publishing, yet unlike about 5 years ago, they now gatekeep a lot of those communications -- many times I post something on my Facebook or Instagram that I want friends to see, and Meta doesn't show that content to my friends, just because it isn't a "Reel". Friends in my age range (+/-10) who grew up in North America tend to use only Meta's products, so I basically feel like I'm talking to a wall most of the time.
A/B testing isn't a new thing - I'm sure the inventor of the wheel experimented with different shapes, and the buyer of the hexagonal wheel probably didn't have the best user experience.
Multiply that by the number of people in the world and the number of products people use, and A/B testing is really up there as possibly one of the most beneficial ideas ever.
I really don't understand those who claim it should be banned - I see no way that testing two different versions of a website with people who desire to use that website can bring sufficient harm to outweigh those massive benefits.
I was in the "B" group, and felt so humiliated at how many times I tried to reset my password to get into Facebook.
* https://www.forbes.com/sites/kashmirhill/2014/06/28/facebook...
It's a great tool, and its impact all depends on how you use it.
The major issue with A/B testing in the workplace is it causes confusion and slows people down when you change things. Which makes these tests really expensive even if they are seemingly easy to preform. So, I would call it useful but flawed.
If you're building a tool to make life easier for the user, something that gives them a better experience is your optimal outcome. This seems like a scenario where A/B can produce a good outcome.
The challenge is when you throw in an ad-based revenue model, and the A/B testing is then optimized for the opposite (eyeball-hours, linear metres scrolled per session, ad spots passed, ads clicked) - engagement-based business models end up (I'd argue) A/B optimizing for the opposite of what their users want, to get them to spend longer doing a task they could have done quicker.
The funny thing is - the ad-based revenue model is not the only possible variant. Last time I’ve checked Facebook’s profits per user were $7 per quarter, that is $28 a year. At the same time I am paying LiveJournal $25 a year for the ad-free version. Just taking my money looks like a much better model in many respects:
- less overhead: a lot of people doing these studies how to force me to look at something I do not want to look at will be free to do something more useful to the society;
- streamlined relationship between me and my publisher: in this model there is no advertiser who can say “I do not like these texts, no revenue for you”.
That’s why I prefer to pay for some Substack authors, like Matt Taibbi and Glen Greenwald, than to try to fish their texts for free amid some sea of “clever” advertising (hey AI testers, I bought this thing already, what’s the point of forcing it on me again and again?).
I kinda wish that Brave model (my money distributed between sites I visited) got more traction. It looks much more healthy.
I bet by at least 4 orders of magnitude and likely 5 or 6.
Perhaps you should just agree that, "not all a/b testing is the same".
Quote - "Speaking from work experience: A/B testing doesn't look into the nuances of usability or productivity, it looks at easy-to-quantify metrics like conversion rates and money spent"
There's also a neat sleight of hand here. Your inventor of the wheel surely tested multiple variants to optimize for the utility of his invention to the user. The A/B testing that's problematic is about optimizing taking advantage of the user. That doesn't lead to better experience, but the opposite. This is what's increasingly popular, and this is what people complain about or want to see banned.
Related: attention economy is predicated on bad user experience, because it makes money from friction.
The primary goal of A/B testing is to see what's more profitable.
If that happens to result in better UI that's a side effect.
In fact, it could result in less usability (relevant to this conversation, it probably resulted in the frustrating "algorithm-based" timeline at FB/Twitter/etc).
Well, I think I'll provide a disagreeing opinion. :)
I assume this opinion probably comes from your past experiences, and I believe it is true in many cases. Since I'm not American and have never worked in an American corporate environment, I can't say what is true over there... but my experience in EU and Canada with A/A/B, A/B/C and typical A/B testing (as well as building such testing tools for others) was not like that.
For example, when building tutorials for users, profitability is far from being the primary objective. Same goes for building documentation, programming languages, open-source software, internal tooling and other such things.
Of course, I get that in the end, profitability is the primary goal of the company (with some exceptions). But I maintain that not all A/B tests have profitability as their primary goal, which makes the previous statement an incorrect generalization IMO.
A/B testing lead to the development of effective "dark patterns" in UI that trick users into doing things they don't want or don't understand, and then making it difficult to undo.
Totally the same as A/B testing button placement. Totally.
Consider yourself that inventor, would you A/B test hexagonal vs round?
As for the other aspects they sound like great targets for testing within different use-cases, but I'm not sure why that'd be an A/B test as we think of them now.
Worse for whom? I feel like a lot of the A/B testing results in more revenue, a more addictive app, and less user satisfaction, because they're not testing for anything beneficial to the user, because at least with FB, you're not the customer, their advertisers are the customer.
For example, I have used A/B testing to see find ways to help users get a task done with fewer clicks, saving them time.
I have not experienced a product become better for the user as a result of involuntary A/B testing in my entire adult life.
Producers and consumer have both an adversarial relationship and a mutually beneficial relationship, and the distinction between these two is essentially the split between voluntary A/B tests and involuntary ones. In the adversarial component, the producer is trying to figure out how to extract more money from the consumer, without improving the product. Alternatively, (and equivalently), how to make the product cheaper, but also worse, in a way that yhe customer doesnt notice (with their wallet). A proactive version of the "market for lemons".
For instance, if you A/B test your cancellation process to minimize the number of people who cancel their subscriptions, you will almost certainly do something that makes you some additional money, and is also unambiguously evil.
Any A/B testing that is mutual benefit to consumers and producers can be done with consent, by volunteers. And the miniscule amount of scientific rigor you would lose by doing so is not worth the tremendous sacrifice we have seen in quality of consumables in the past 2 decades (probably longer, but i do not have the personal experience to go longer)
You might be compelled to describe involuntary A/B testing as a strategy for maximizing evil subject to the constraint that it be legal, but it often dips its toes into seeing what is illegal but still profitable, and is capable of fundamentally undermining our legal system and even our political system.
The technology has grown more powerful. The addition of computers that can optimize essentially arbitrary objective functions has serious existential implications for humanity.
A blanket ban on the practice, incurring the total dissolution of any corporate entity found guilty of the practice of involuntary A/B testing, would be a start.
If you did, how would you know?
When craigslist added the map that shows you where all of the people are offering the thing you are interested in. That was a very good change, but thats pretty far from how craigslist operates.
When dominos stopped serving hot glue on cardboard, its pretty easy to see how that didnt come about by furtive A/B testing. They were pretty confident people would like the new pizza more than the old pizza. So they told them about it. Boy did that work for dominos.
That actually speaks more generally to my point. If you're making a change that you think people will like, you tell them about it, because even if it turns out that they dont like it more, the fact that they thought they would and you did it generates quite a lot of good will for them.
Do we live in the same universe? As far as I can tell, software keeps trending worse. Usability is terrible, options and settings keep getting moved around and hidden, software is less responsive than it used to be...
But the majority of people choose to use a modern computer, presumably because they find it overall more useful than their old MS DOS computer and software.
Sure - there are gripes, but they must be outweighed by something pretty big for 99.99% of people to choose a new computer over a 30 year old one.
Imagine thinking that seriously.
To me the latter view is the one that’s hard to take seriously.
Within a university, research with human subjects is required to pass an ethical review before it is allowed to proceed. Given the scale and impact of the research conducted by Facebook on its users, it is entirely reasonable to hold them to the same standard.
So we agree that ab testing is good for optimizing toward a numerical objective. You then seem to think that either:
A) There are simply no numerical objectives that correlate with good ux or
B) Every software company ever is optimizing toward perverse incentives by which they take more money from their users while making their products worse
It’s probably B that you believe, and this is such a myopic and paternalistic view. There are a couple cases where it’s a problem, eg cancellation flows. But this problem is orthogonal to AB testing (try cancelling your newspaper subscription in 1994). AB testing is mostly just trying to improve the rate at which people sign up or buy something, and in this case, your objection hinges on the hidden premise that people are idiots.
> Within a university, research with human subjects is required to pass an ethical review before it is allowed to proceed. Given the scale and impact of the research conducted by Facebook on its users, it is entirely reasonable to hold them to the same standard.
One man’s ponens is another’s tollens. See: https://slatestarcodex.com/2017/08/29/my-irb-nightmare/
My qualms are not so much with the method as the morals that guide it. It's agnostic but when operationalized in a faulty moral framework can definitely lead to bad results.
Read the whistle blower report, witness the evolution from seeing content from your friends posted in less addictive chronological feed to addictive content your friends like in the internet sorted by addictive news. Hell, the site started as a PHP hack to creep on pretty women. They sold a bunch of data to foreign adversaries. For years, they let people sell ads to Nazis. They don’t give the people faced with the psychologically brutal jobs of moderation get benefits. They have been a platform for genocide and government surveillance.
I might be missing some examples of them missing some money to do the right thing, but nothing comes to mind.
Absolutely, go bananas A/B testing different colors for a “Sign Up” button or testing different pricing models.
But let’s not go bananas optimizing algorithms that are damaging to users mental health at a massive scale.
In research, these types of experiments typically require consent..
obviously "A/B testing" is not "hiring psychologists", because one is in the category of experimental methods the other one is about human resources
yet there are obvious connections. A/B testing is used to increase "conversion", which is profitability. which is the same fuckin' thing as addictiveness in case of a site where you pay with your eyeballs
That is untrue.
A/B testing is not the same thing as seeing how bad your users' mental state becomes if you muck with what they read.
A/B testing is a tool. It can be used for good or evil. That was Facebook's choice how to use it.
They did it to make sure they could keep people's attention, despite the psychological damage.
(This is not an attack on you or your otherwise valid point. Just a reminder that people should be mindful of their ethical obligations to get informed consent and not cause harm to others with their experiments).
Now if you had said “fuck meta because my feed is filled with an unintelligible mix of baby photos and political screeds,” I’d totally follow.
"On 6 December 2021, approximately a hundred Rohingya refugees launched a $150 billion lawsuit against Facebook, alleging that it did not do enough to prevent the proliferation of anti-Rohingya hate speech because it was interested in prioritizing engagement."
https://en.wikipedia.org/wiki/Facebook_content_management_co...
I feel engaging in commerce in any way at all is impossible without being "evil" under this definition.
Aren't advertisements at their core unconsented psychological manipulation? What about retail store design? Is providing customer service altogether just manipulation?
I think I take issue with the word "evil." It seems to imply a certain malice or intent to harm, which just isn't logical, given the context.
And no, I can't give you a clear brightline test right now to determine this. It takes homework.
But.
If you want to A/B test whether people prefer blue chewing-gum to green, be my guest.
If you want to A/B test for literal depression and sell stuff to people based on that, you're probably evil.
The line's somewhere in between there. Now we know where Meta has chosen to go on some of this; and again, this is from WHAT IS KNOWN TO BE PUBLIC.
The safest thing is to assume the worst, then. May they rot.
"They" is unnecessarily vague I guess. It's not fair to write off the entire population of Facebook staff... There are surely a substantial chunk of employees who were entirely unaware of these tactics.
It's easy to claim an organization is evil. Less so for individuals, and with good reason. There's something to be said for remembering to humanize people, even those most "undeserving."
There is no harm in me saying out loud "Facebook is an evil company" even if I'm completely wrong, because they are so powerful. It would valuable for everyone to scream at Facebook at being evil because that would force them to show and prove that they are not.
We shouldn't get stuck on being "accurate." Throwing rocks is not just okay, it's a good idea if it forces the powerful party to act/respond.
As for the individuals -- it's their job to defend the company. If that hurts their feelings or whatever, too bad. I don't care and neither should you (presuming that Facebook is potentially very harmful.)
Yes. 100% yes.
> What about retail store design?
Also yes.
Advertising exists to try and make you purchase product “x”, regardless of whether you need it, or it meets your requirements. Even “harmless” display advertising exists to gently push to mind into forming the desired emotional association with a product.
Store layouts are well documented for being deliberately anti-efficient: they make you walk past everything else to find the things you need, with the goal of exposing you to more advertising and product placement. Combine that with strategies like putting sweets and other “low friction” products at the checkout, where you’re more likely to make a spur of the moment emotional decision under pressure, or be hassled by your children for sweets, and you have something that is inherently exploitative and morally questionable at a minimum.
Evil is probably too high-modality, but unethical and exploitative are definitely suitable.
I only took issue with the choice of "evil" because I am working on curbing my own hyperbole and sweeping generalizations.
Nothing wrong with calling "Facebook" evil. It's not a person, it's an artificial entity. It can take it. (I too would be more reluctant when it comes to people)
If a store owner puts two different candy bars at the register to see which one sells more, is that an evil thing?
When someone tries out a different way of greeting strangers at a party, are they being evil? What if they write down the results?
But humans are complex beings, and it's just realistic to view us all with some nuance rather than casually tossing everyone into good/evil baskets. The gp outlined an example of decent behaviour, and it remains just that regardless of our attitudes on other grounds towards Zuckersnuffles and/or Meta. Even murderers can be kind and decent people given in some contexts (witnessed at first hand).
With that in mind, I'd like to point out root merely expressed an anecdote about above and beyond humane treatment of employees from the CEO. Parent described the hacker-friendly culture. Those appear to be first hand accounts.
How does your comment relate contribute to that discussion?
Further, how does over the top and genericized outrage and harm wishing on a cohort of innocent people advance society in any meaningful ways?
All I'm saying is that the harm they inflict on billions of people outside the company outweighs the good they do for thousands inside. Just because there's both good and bad doesn't mean the degree of each cannot be weighed at all.
I asked a question, no conclusions drawn. If I had a point, it's that the comment was off topic with regard to the comment it replied to, and warrants its own tree of discussion. It was a rant; There wasn't even an attempt at a segue or good faith effort to provide contrast.
What "gives me the right to decide how the discussion is framed" are the hn guidelines (which you violated yourself by presenting a strawman).
Consider how this may have helped you to develop your opinion.
The Cambridge Analytica thing probably didn't have a meaningful impact on the elections. Facebook was dumb to allow partner apps access to so much data and rely on those partners to follow Facebook's policies, but they recognized the mistake and probably overcorrected.
All the other coverage probably serves to reinforce and amplify negative sentiments about Facebook, and a ton of it wasn't deserved. People cite the Rohingya, I remember also a news cycle about Facebook profiting from hate speech.
Those things happened, but Facebook had also built probably the most expensive and effective hate speech filtering operation in the world. That it be 100% effective is not a reasonable goal. With billions of pieces of content even 99.999% effectiveness will result in examples that the press can point to reinforce a narrative about Facebook profiting from hate speech.
I doubt there ever was any profit in serving hate speech. The ad revenue from filter misses would not have been big enough to pay for the filtering operation itself.
That's a pretty big claim to make without evidence. At the very least I think we can agree that political parties wouldn't be investing so much money into social media campaigns (analytics, marketing, etc.) if they didn't think it was impactful.
https://towardsdatascience.com/effect-of-cambridge-analytica...
> Sumpter analyzed the accuracy of Cambridge Analytica’s regression models in his book ‘Outnumbered.’ He used a publicly available dataset created by Michal Kosinski and his colleagues, a psychologist, who created an anonymized database of 20,000 Facebook users. Of the 20,000 Facebook users 19,742 were US-based, and of that amount 4,744 had registered their preferred political party Democratic or Republican, and had also liked over 50 Facebook pages. Sumpter first aimed to test the accuracy of regression models in general, and so created a model which predicted political party allegiance based on Facebook page likes. He concluded that the regression model worked “very well for hardcore Democrats and Republicans” but “does not reveal anything about the 76 percent of users who did not put their political allegiance on Facebook” (Sumpter, pg. 52–53). He also describes how just because the model may have revealed, for instance, that Democrats tend to like Harry Potter, it does not necessarily mean that other Harry Potter fans like Democrats. Therefore, a strategy employed by Democrats to aim to get Harry Potter fans to vote, may not necessarily benefit them.
You'd probably have better spent your money by using party-owned voter databases to create custom audiences based on email addresses. Everything I've seen points to Cambridge Analytica being mainly an operation to grift campaign dollars. From the same article:
> CEO Alexander Nix himself corroborates these results. He claimed in his testimony to members of the British parliament’s ‘Digital, Culture, Media and Sport committee,’. He contended that Kogan’s dataset wasn’t very useful and that made up a tiny part of their overall strategy for the 2016 United States presidential election. How do we resonate this admission, with his presentation at the Concordia summit, in which Nix openly bragged about the ability to wield Facebook data to tune an incredibly powerful instrument that significantly impacts elections? The answer to that came from Nix himself in his testimony, claiming that he has in the past used hyperbole when pitching his company to potential clients. This view is corroborated by Kogan who mentions how “Nix is trying to promote (the personality algorithm) because he has a strong financial incentive to tell a story about how Cambridge Analytica have a secret weapon” (Sumpter, pg. 54).
There is plenty to lay at the feet of Meta, but I've always thought the CA scandal was an overreaction.
I have seen many people brainwashed by highly targeted ads and conspiracies that found them on facebook. Those people vote, break into congress, etc.
deleting your individual account doesn't change that
It shows, that a life without FB is possible. The more people do it, the easier it gets.
Until the network effect is overcome.
1. https://www.nytimes.com/2018/10/15/technology/myanmar-facebo... 2. https://www.wsj.com/amp/articles/facebook-hate-speech-india-... 3. https://www.wsj.com/articles/facebook-services-are-used-to-s... 4. https://www.npr.org/2021/04/08/985143101/stop-lying-muslim-r... 5. https://theintercept.com/2019/12/07/facebook-mark-zuckerberg...
I think maybe your beef is with the nature of technology today and/or our current culture that enables/glorifies it's mass use. What's the solution?
[1] https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb...
Link pls
This is a really malicious way to describe showing two users the same button but in different locations