Should There Be Limits on Persuasive Technologies?
schneier.com
schneier.com
---
If we know that humans have all sorts of cognitive biases, how come it's ok to use that fact while at the same time we insist there's some kind of free market?
Say you discover that putting good-looking women next to cars causes the sale of cars to increase. Why does nobody question whether it is legitimate to do so? It's as if there's a line between actively lying ("Studies show that men who buy this car will find many many women attracted to them") and just putting it there suggestively, for some as yet undescribed but working cognitive bias to do its magic.
Some advertisers even make a joke out of it, eg the Lynx ads where the dude is thronged by a huge horde of women. It's a cliché, for a good reason.
I suppose most people will just say you have free will and it's your own fault for thinking what was suggested, but I sense this is more of a grey zone than most people are willing to admit. How can the free market work if everyone is so easily affected by suggestion?
---
Of course this also applies to the free market in ideas. In what sense are people free to make up their minds if it's decided for them what they should see, whether or not the government is doing it or FB? Isn't this the same as the authoritarian nightmares that we've been pointing fingers at?
If you start making certain things illegal to say, you can use that against people. For example, given enough money for lawyers, you can sue people for saying that "these cars don't get you women" using the same anti-free speech regulations by finding holes and exceptions in them. History has shown that lawyers can pull this off.
Yes, they are.
We wouldn't lose much if advertising would just be forbidden. A traditional definition of it would probably be enough to get rid of most of it. The main problem is that huge sums of money are involved, and almost all media profit from it to some extent, so there's a huge incentive to shut down this conversation.
As long as people care what is popular and fashionable rather than what is good there will be ways of exploiting it.
Also, if people found themselves regretting purchases of products and services recommended to them by a particular review site, they would be free to use a different review site in future.
Hopefully these sites wouldn't operate on a non-refundable yearly-subscription model, making people reluctant to "waste" the money they had spent. The business model for such sites might be a little tricky, since they shouldn't really be making money from affiliate fees, and web advertising wouldn't exist in this world either.
This is the crux of the question. I don't consent to any of it, but just walking down the streets I'm targeted by an assault on my brain from every direction. Ditto picking up my phone or browsing the web (were it not for uBlock). It's not ethical.
As for review sites being shilled, mandate big disclaimers detailing sponsors (shill praising product X would have to have a big banner saying "this content paid for by X", as its already done in other contexts). Reputation would eventually form around trusted reviwers.
We would lose the current ad industry and just replace it with some other form of it.
You can't get rid odd advertising, because the existence of the product itself is already advertising. Let's imagine that advertising were banned. How could a car manufacturer still advertise their cars? Put bigger logos on them and sell lots of cheaper models for a while. That way the city will be full of your vehicles and any time someone sees a car they'll think of your cars. Same goes for Coca-Cola and other brands. In that case they just need to take up more shelf-space.
Bingo. And that's why normal people don't care about ads if they get Facebook and Google for free.
Information exchange can take place when people use their imagination to create with raw materials. Stock stores with raw materials. Focus public governance on health, not public-private collusion.
Think like open-source; take computer, make it useful to you.
We’re circling last generations emotional model for packaging given their logistics capabilities.
IoT can act as a model here; smaller uni-task gadgets or general compute gadgets, needing less code.
The sunk cost fallacy at scale is how our system works now. That simply props up the ingenuity of those who got there first.
Why the ever loving hell do we need teachers with high education bills and masters degrees? Were we not able to reach reading for the thousand years before?
This social monolith building needs to billionaires is merely following a social monolith to kings and priests.
We move around atomically but pay a tax on our effort upwards. The result? Decades of inequality growth, deflation of buying power.
It’s not technical change we need but emotional; we don’t owe these people deference.
Pre-packaged experience is no different than “here’s your Bible, Timmy.”
For example, if a concert promoter can't buy a 30 second spot on local radio to promote and upcoming show, they will instead pay for the band to be interviewed by a DJ where the interview is little more than the band talking about the upcoming show. I don't think that's an improvement.
Precisely. Spec sheets, unbiased reviews, ratings systems that vet purchases, those add value. Almost by definition, what we consider advertisement is designed to distort humans from making rational free-market choices. Stretching the analogy: Attention is finite, therefore every ad you consume moves you further from perfect information.
That's a purely pragmatic argument without getting into the whole bit where ads use all sorts of psychological tools to shape behavior.
How about just making it no longer tax deductible as a start and see how that goes? Spend some of the money on increased consumer protection.
I can't argue with that, but the alternatives are worse. If you centralise power, sooner or later the advertising exec gets control of the powerful body, and now you can't choose to resist even if you can see that what is happening is bad.
A key part of the free market is precisely that the world is actually quite predictable. The fact that people sometimes make predictably bad choices doesn't especially undermine the free market. The market doesn't require people make good choices, it just redirects resources to people who make better choices than the average.
The boundary between overly enthusiastic promises and actual fraud is one that's in different places for different jurisdictions and is constantly at the forefront of litigation as people invent new fraud schemes.
> "People don't act randomly. They have reasons for what they are doing. I think their reasons are bad".
Sometimes the reasons are bad in very objective ways.
A goes down to the market to buy a kilogram of apples. A vendor B advertises 1kg apples for a pound. He weighs them out on his scales. A hands over his pound. When he gets home he finds he has 800g of apples. Was A's reason for purchasing the apples from B good or bad?
(laws against short measure have been a thing since at least Roman times; I believe they also had a few product quality laws, although the canonical example there is always German beer law from 1516)
1. Philosophical - you don't want to get the threat of physical force involved unless there is a very very good reason for doing so. That is, you don't want to make things illegal. So we forbid outright lying and fraud, openly misleading customers, but allow things that you describe. If you have a private online platform, why should you not be allowed to or be compelled to moderate its content? Yes, tech companies will try to maximize user engagement, but what else should they be trying to maximize? It is up to the rest of society to develop a proper culture and information hygiene to shape their demand, and tech companies will adapt. It can develop organically and passed from generation to generation, or you can try to expedite the process through schools, media and other mechanisms that exist in a society.
2. Practical - people tend to get interested by forbidden things, and they seem to like their biases. In the Soviet Union, for example, perhaps most people were fascinated by the Hollywood movies of the 70s and the 80s, and even the way the Western brands were being advertised. They seemed 'cooler' to the Soviet products that were mostly devoid of marketing. Try to eliminate most biases, and people will vote with their feet. Arguably a better approach is to allow things, but in good faith educate people on how to best deal with them and why that is the right way of dealing with them. Same way a lot of parents today would explain to their children why they should stay away from cigarettes, for example, or wash their hands before having food.
1. Are the freedoms of corporations to act without the coercion of the state more important than the freedom of citizens to act without the subtle but effective coercion of corporate persuasion? The case for individual freedom doesn't always imply the absence of action by the state.
2. Would restricting manipulative practices by businesses really lead to more businesses exploring those practices? That may be the case in restricting consumer choices but would it also be true for corporate behavior? I'm not so sure.
I also think that calling persuasion 'coercion' is misleading. Coercion implies the threat of force or violence. You and your family still have the choice of switching off your TV, not using facebook or twitter, living your life as you want. Assuming you can live in a community of like-minded people such that there are few social costs of doing so for you and your family. I would like to see more such communities, especially among people working in tech.
Restricting some practices by businesses would lead to foreign businesses and cultures becoming more attractive to consumers.
I don't want to put so many restrictions on small business, because they're more directly driven by individuals (although if their behavior starts having a significant negative impact on individuals, then perhaps that can be investigated). Small business has far less power to negatively impact individuals when compared to multi-billion-dollar multinational organizations.
I believe that society should favor and protect the freedom of the individual over the freedom of the large corporation. Without those individuals, society is useless. Of course, that's sidestepping the issue that society far too often sticks its teeth in where it doesn't belong, and won't bite down where they'd actually be useful.
Your last sentence is honestly a copout (not on your part personally, but in general as an intellectual idea). That basically embodies a race to the bottom, as countries around the world erode freedom, and the rest of the world "harmonizes" so as not to be disadvantaged on the world stage. You see this now with the current copyright situation.
It's kind of the same idea as the GPL protecting the greater freedom of software by restricting some of the freedoms that a developer may wish to have. Many similar arguments can apply too, as for example I've heard it said "GPL's restrictions will just push people to use less-restrictive licenses" That may be somewhat true, but there are also people who value the greater freedom more than their personal freedom to use the software in a non-free manner. Note that this isn't an endorsement of GPL over other licenses, just a parallel to be drawn that is somewhat relevant to HN.
We are sort of approaching a standard argument between classical and modern liberals, so it would be pointless to just repeat the same tired arguments. Having said that, I would love to see more of thoughtful debate between classical and modern liberals. Usually you tend to see both sides just preaching to their choirs or a very shallow debate at best.
Many of these laws are written in blood.
So no, legislation that limited large companies ability to exploit the heuristics and biases of their customers would not be 'strange'. We can also legislate against foreign companies.
This is an argument to scrap OSHA and allow slavery. How else, after all, can businesses be expected to compete globally?
These also apply to individuals.
Enough quantitative change becomes qualitative. Ice is just a bunch of water molecules just a bit differently organized. Graphite is worthless, even though it's the same atoms as diamond, but differently organized.
Many people joining together into a corporation can become its own thing, acting as if it had "goals" on its own that may be not in line with the goals of the individual participants. Corporations are real entities separate from individuals. It's emergence. It's a "socially constructed" entity but no less real.
Yuval Harari describes this very convincingly in Sapiens.
Private investigators - not sure how legal that is around here and what are the constraints.
Example - for a job application I recently needed a background check by a US company. As soon as I filled out my current location as a European country, a consent form popped up. They can't go sniffing around even in the public records and compile stacks of background info on people at the scale a company does this.
Lots of things are public outdoors only with the common sense understanding that it's ephemeral, is just remembered by people's imperfect memory and won't be meticulously categorized and stored somewhere, unless there is consent. Now, you can make small scale notes and a personal journal about whom you got to know at a party etc. You can take tourist shots and have random people in the pictures but you can't follow people around and photograph them specifically as the main subject of the photo without their consent. Public figures and officials like the police don't have such privacy rights on duty.
It's about the scale of it, and common sense judgment of whether it systematic data collection or something just happening normally as life goes on. But I'm not a lawyer, I'm just describing the broad principle.
The law and people's intuition may be different in the US.
----
If you don't like this, think about another example: hitting/patting your friend on the back as banter or congratulations or something. How strongly do I have to hit them to commit assault? What if it's my frail grandma? Intent, scale and common sense matter. There is no law saying the exact amount of force I am allowed to apply.
I think this is it precisely. I agree with you that intent, scale and common sense matter. Some of this can be included in the law or left up to a judge/magistrate to decide.
Going back to the premise of the article, before we design a privacy law for targeted advertising, we have to agree on what constitutes a level of manipulation so great it effectively takes agency away from those targeted.
No. Corporations are a creature of government policy, and receive entitlements that sole proprietorships or general partnerships do not receive, such as limitation of liability.
Don't butcher the English language.
That's not what coercion means. Persuasion is not a "coercion".
Just as the environmentalists say that you can't really throw things away because there is no "away", increasingly there is no "rest of society" that exists completely outside online culture. Especially in the year we've all been quarantined inside.
However, ads or fake news article that takes many exposures to reprogram you (by using the cognitive bias back channels as you mention) are perfectly fine. ¯\_(ツ)_/¯
The other point is that with ads and articles you have to engage with them yourself. Something can grab your attention, but you can decide to ignore it anyway.
Instead, we look at the total effect (and often the intended effect) of multiple small changes, and decide whether they are a net positive for society.
With tobacco it's far easier to show a correlation, because people can just not smoke. People can't just avoid the effects of advertising, because it's all around us. Even word of mouth can be an ad campaign.
Also, tobacco probably is too regulated. It should be regulated to stop other people from experiencing the negative effects of somebody smoking. Limiting what flavors cigarettes can have and enormously taxing them seems like it has gone too far. People should be allowed to make their own choices in a free society.
Thinking out loud, I would be curious to see a study on people who play 'Loot Box/Gacha' style games. Include both people who have been playing for a while, as well as people who 'start' playing during the study. Include non-players.
Measure their other 'vice' behaviors (i.e. Drinking alcohol, nicotine, real gambling, gentlemen's clubs, etc.) over the same time.
My hypothesis is that in those who are introduced to Gacha games, you will see an increase in other 'addictive' behavior, because the content they see will whittle away at their resistance to other addictive behaviors.
I understand what pattern you're going for, but I don't think the study would show that. The number of people that play gacha games is small conjugated to the number of people that smoke, drink, gamble etc. Chances are that you'd likely see the reverse - doing those things could increase your likelihood of playing gacha games (or well, spending money on them - it's entirely possible to play most of these games as virtually free).
When the rest of the "free society" has to bear the costs of certain choices, it doesn't seem so illegitimate to argue that it has some interest in constraining those choices.
And there of course is the endless contradiction of being a social creature: on the one hand, being part of a society creates possibilities that were not there before, but also leads to constraints that were also not there before.
Endless political philosophers across centuries have grappled with how best to resolve this paradox, with no definitive answer at this point.
But we obviously tolerate this to some extent. I think it's reasonable to limit smoking so that it doesn't cause second hand smoking effects on other people. However, going further than that seems odd to me. If they're only hurting themselves then I don't see the problem with it. We allow people to do extreme sports, we allow them to drink alcohol, we allow them to do high stress jobs, we allow them to eat sugar, we allow them to do a long list of things that can hurt them, but we tolerate most of those because people should be free to choose how to live their life. Yet somehow on a small list of things we control people's behavior. If they don't follow it then they're considered immoral. (We don't call them immoral, but that's how we treat them.)
Just like laws protect the physically weak from exploitation, they should protect the psychologically weak from psychological exploitation.
You wouldn't accept someone holding you at gunpoint demanding your money, and you would have no choice but to give up you belongings. Even though it was your choice to give up your stuff, you were coerced into it. gacha/loot boxes and similar addictive phenomena pry on your psych.
This isn't some conspiracy, this is a tried and true method which you can learn about from the whaler himself[1].
That's a slippery slope, when we remember that researchers are constantly looking for the right hooks that edge the 'persuasion' closer to entrapment.
Doubly so when we consider neuroatypicals; While many people think of a 'seizure' as something that results in shaking movements/etc, There is a nonzero size of the population that can have seizures that are more 'internal'; the stimulus results in either a locking of focus on an item, or a sort of disoriented haze (at which point, for some, their subconscious is wide open.)
Those who take the time to study hypnotism/NLP and are more wise than intelligent understand that we are playing with fire here folks.
I disagree. It's been shown[0] that repeatedly exposing people to information makes them more likely to believe it's true. As a further example, try to not think about your breathing after reading this sentence :)
[0] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2958246
Associating emotions to a product or getting attention is an old trick that existed before modern marketing. The smoking example can be generalized for fashion and there are mechanisms like peer pressure involved that certainly incentivize consumption. But they are not overriding your will. Addictions can, but even then I would say there is still a free will involved.
> but I sense this is more of a grey zone than most people are willing to admit.
I do believe ads affect me, but the scope is limited and regulation would be more draconian than my natural inclination to give products an unjust bonus for boobs. However, it may not work, because I come to the conclusion that the product must lacking if you try to sell it with dirty tricks.
> Isn't this the same as the authoritarian nightmares that we've been pointing fingers at?
No, because people can make up their minds, otherwise they would have a lot of cars by now. They loose that, however, if you regulate too excessively because the decision is already made for you. There are sensible reasons for regulation, so it is a gray area, but I don't see it as helpful here.
Empirical counter evidence in favor of my free will for any practical purpose: There is no irresistible ad.
Of course I don't think blandifying ads is the way to go. Making ads attractive is a cultural phenomenon.
An implicit suggestion of fame, class, social status is fine. It's part of the game. It's an implicit suggestion in the end.
But everything has a limit. What might be acceptable in an adult ad might not be so acceptable for a teenager or a kids ad.
More worrying is propaganda, which is misleading/false in non-obvious ways and not explicitly an ad.
Taking things to the extreme: maybe people are by default violent and will kill a few people in their lifetime to get their way. But the culture counterbalances this.
Likewise everyone old enough to have a sex drive has seen enough publicity with barely-clad attractive women to know the ruse. Of course, people are still vulnerable to these things (cf. onlyfans, etc.) but at some point you have to establish that the rules are clear and if people still want to indulge the fantasy of a beautiful woman by buying a car, well, whatever.
Sure, we still put backstops to this with drugs, gambling. But the use of sex as a tool for attention grabbing is way too wide a net to cast.
Clearly not, for the simple fact that they do it anyway. It works, else you wouldn't see it in ads.
(The following video analyzes a fragment of the movie "West Side Story" and it's fully SFW)
Mild amusement can be had from the example cases in the sex section: https://www.asa.org.uk/advice-online/offence-sexism.html
(I note that vaunted free speech zone America is actually more restrictive in what sexual material can be shown on TV or even marketed; there isn't a US "Babestation" TV channel, is there?)
TV coverage of the yearly five-days Mardi Gras has also dialed back very very significantly over the past decade. Internationally we can see the case of F1 "Grid Girls". The culture sometimes puts backpressure on this.
(Societies have odd contradictions on sex generically. In relatively relaxed, women-breastfeed-in-public, close-physical-contact among strangers Brazil, topless sunbathing is not only frowned upon, it's illegal. People will snitch on tourists and the police will go and arrest them.)
Or perhaps it is time to admit that talks which are drawing inferences are just talks, and while my sisters were asking dads to draw horses, I was asking dad to offer me a drawing of a powerpoint, because things that plug to each other fascinated me.
If sex creates differences in the body, it is only stunning ideology that draws us to affirm it creates no difference in average in the brain, and that everything is socially constr... No, I’m not James Damore.
I asked if he had other children. He had a daughter who was eight. He was not doing the same things for her.
Enrollment among women in computer science was growing steadily until the 80s, when it turned around and declined. Our biology has not changed during that time frame. And the core job of a software engineer hasn't really changed at that time. Other engineering disciplines did not experience the same change in trajectory. Innate biological preferences have a hard time explaining the unique trajectory experienced in software.
You don't just need to demonstrate that there exist differences in the brain between men and women, but that the specific differences lead to the observed population-wide outcomes.
People can make poor choices because of cognitive biases, or they can have choices made for them by other people with cognitive biases. The other people can be unelected, unaccountable leaders, or leaders that are chosen by voters, and politics seems to be where cognitive biases are worst.
In general, I would rather suffer for my own cognitive biases than the biases of elected officials and voters, but that's not to say I advocate for free markets in every scenario, because there is a lot more to consider than individual choice in that discussion.
The point is, people don't buy things for their own sake. They buy them for what they think the thing can do for them. Any car will get you to point B, but some people will go for a cheap, utilitarian car because they want to save money (or maybe the utilitarian aesthetic is their thing), while others will go for the flashy car for that feeling of sex appeal (even if women don't suddenly fall all over a new car owner, the feeling of confidence is a social benefit to the buyer, even if that's not worth the asking price).
More generally, though, all communication has this sort of color to it. We see anti-privacy legislation being touted as protecting children and fighting crime. Small talk is not really about sports. So I don't think it's realistic to legislate persuasion. I would probably be behind making formal logic a part of public school curriculum, though, so people are better equipped to discern for themselves when persuasion they're exposed to is nonsense (among other benefits).
In math, would you market a more fuel efficient car by telling the percentage difference in miles per gallon or liters per 100 km?
These are obviously just some examples, but the point is that critical thinking is a cross-cutting concern and cannot be segregated into one class. It should be constantly invoked. Instead of listing fact X, explain how people figured it out. Why did people not see it before and what did they think instead? Was it universally accepted immediately or what were the criticisms?
The thing is, though, this is extremely difficult to teach. You'd need extraordinary teachers with lots of background knowledge, since such debates and brainstorming can quickly go astray of the standard curriculum and quite easily end up in difficult and perhaps unknown territory. Perhaps you could do this with an Internet connected laptop and a projector, diving into Wikipedia rabbit holes in real time. But this would be different each year, and you'd have no way of ensuring a standard curriculum. If you don't make things required for the test, many students just won't listen, no matter how engaging the class may be.
Such teachers would demand higher salaries to be competitive with places that can also make great use of the limited supply of critical thinkers. Maybe it's offensive but teachers as a demographic aren't that smart in most countries,since the requirements are pretty low to graduate and the social reputation of the profession is also quite low.
But even if you find great teachers, it's even more difficult to test, as it's open ended, not objective and not standardizable.
I doubt pretty girls in ads are actually supposed to mean the goods they advertise make a man more attractive. Who (except teens) would believe that, consciously or subconsciously? You just get attracted yourself and that's enough, simply seeing a pretty girl fires the hormones and neurotransmitters making you feel good about what she advertises, no semantic load necessary.
You wouldn't think eating candy bars will make you athletic. But they market it with athletes and active people playing soccer etc.
They don't show some fat dude sitting in the dark in front of the computer shoving candy and chips and coke in his face and becoming diabetic.
The associations can be immediate if repeated often enough. When picking a product you don't reason "okay this will make my life X", but you feel a familiarity, a draw, a positive emotion. Not necessarily consciously. But especially after a stressful day, in the supermarket you will be more prone to emotional, autopilot handling.
How about chewing gum and mint ads? Many try for messages that are essentially "don't get cock-blocked by your bad breath" or something similar. And they work.
https://pdfs.semanticscholar.org/c86a/142c0ae2d70d95f6292af6...
And this has changed me so much. Even seeing that stereo typical hot-rod style cars that you are clearly described as attracting women - I actively dislike people that show off even. I have never had an FB account.
I don't know where I'm going with this but yeah, cognitive biases and such and controlling people. I guess just trying to say, it doesn't work on everyone - not identically and as expected at least.
Of course humans aren't perfectly rational but I'd argue it's still a good assumption to make, as a society, because the alternative leads to a very disturbing path. Ultimately, assuming individuals don't really know what's best for themselves can be used to justify all kinds of authoritarian measures from limiting speech to straight up enslavement.
Idiot farmers, who are not educated and don't see reality for how it is (only shadows on the wall), cannot be trusted to run your state. Only those who see reality for how it is (those who are dialectically educated in a school) should be anywhere near power.
I think that even today, this notion of "you are low/stupid therefor we should not listen to you" is almost everywhere in the west.
Banning advertising would kill these products, because they add no value except to solve the problem they created.
Human beings are not computers. No aspect of human behavior is perfectly logical or rational. You simply cannot ban emotional appeals in principle, because all appeals have an emotional component. This way lies a dystopia far more terrible than "grandpa shared some nonsense on facebook".
True enough. But are you implying (not saying you are, just asking) that ~"nothing can be done"?
"Banning emotional appeals....This way lies a dystopia..." is just one option - might there be other approaches that could improve the situation?
> I cant think of any downside here.
I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it.
In my experience, once you give people the basic toolkit for introspection, it can kick off a positive feedback loop. Not everyone will become a stoic, but they can make meaningful progress on their own.
> I can imagine many people who may see a downside of a population who is broadly capable of skilful critical thinking. I don't think it's an accident that we only talk about the crucial need for it, but never talk about actually doing something about it.
On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way
No doubt it is an improvement over baseline - but do you have any opinions on this specific notion that skills in abstract thinking mode may not successfully transfer to object level thinking, particularly when under pressure (ie: conversations)? I truly believe that this is a real phenomenon.
> On the contrary I think those who see this as negative are few and far between. Democratic power in the western world has never been greater than today, so if this is something the people want, there is very little that can stand in their way.
Oh, I think this is a very small group of people. I believe the general public has wanted a lot of things for a very long time (general welfare for all people, within reason, especially domestically), but improvements along these lines seem rather marginal, despite essentially unending productivity and GDP growth. Something seems rather imperfect here - the advertised benefits of democracy seem more impressive than the results, not unlike TV commercials for a Big Mac Meal versus what you actually get at the restaurant.
I think the lack of progress is due to a fundamental misunderstanding of how democracy functions and drives progress. Meaningful progress is only made when persuasion occurs. Living in political or philosophical silos then going to the voting booth only perpetuates the status quo. To drive forward, you have to change someone's mind. Just about anyone can double their political impact on an issue by going out and having a 1 hour conversation with someone who is indifferent on the topic. The biggest trick ever played on the public is telling them they are disenfranchised and that the system is rigged. If people spent 5% of the time and effort they spent lamenting corruption or political advertising on persuasion, change would advance at a rapid pace.
I assume you mean people's perception of the lack of progress. Actual physical progress is independent of the public's perception of it, I would think.
> Meaningful progress is only made when persuasion occurs.
Makes sense. I pay fairly close attention to the variety and nature of political persuasion, and I find it rather....interesting. I think a fair argument can be made that it is often exerted in directions that seem inconsistent with the general well being of the population.
> The biggest trick ever played on the public is telling them they are disenfranchised and that the system is rigged.
I agree that that is "a" trick, but I don't agree that it is the biggest one, by a long shot. Actually, I think there's more than a little evidence that suggests there's a fair amount of truth to it.
> If people spent 5% of the time and effort they spent lamenting corruption or political advertising on persuasion, change would advance at a rapid pace.
I can't imagine how this could be true. Who would be the parties being persuaded in this scenario, and how would that exert influence on the amount of change?
And then looking at it from another perspective: compare the change in well being of the average person in China versus the USA over the last 10 to 20 years - does the above explain the entirety of the differential? Or, might there be some other variables in play here?
>I assume you mean people's perception of the lack of progress. Actual physical progress is independent of the public's perception of it, I would think.
I totally agree.
>> Meaningful progress is only made when persuasion occurs.
>Makes sense. I pay fairly close attention to the variety and nature of political persuasion, and I find it rather....interesting. I think a fair argument can be made that it is often exerted in directions that seem inconsistent with the general well being of the population.
I think it depends on what you count as political persuasion. I think it is more than adds and media. It includes conversations at dinner tables and in lunchrooms. It includes school curriculum, protesters, and door to door political canvasing. I would agree that political advertising and media is often inconsistent with general well being. I think that it is disingenuous and often exerted to benefit a small subset of the population, either corporate shareholders or individual politicians. That said, in my opinion, the influence of advertising and media is relatively small in comparison to the other factors. Sometimes, like a swing voter, it might sway the public a critical 1%, but collectively, the other forces are far more influential.
>> The biggest trick ever played on the public is telling them they are disenfranchised and that the system is rigged.
>I agree that that is "a" trick, but I don't agree that it is the biggest one, by a long shot. Actually, I think there's more than a little evidence that suggests there's a fair amount of truth to it.
See above
>> If people spent 5% of the time and effort they spent lamenting corruption or political advertising on persuasion, change would advance at a rapid pace.
>I can't imagine how this could be true. Who would be the parties being persuaded in this scenario, and how would that exert influence on the amount of change?
The ideal parties to persuade are those with weakly held views. 40% of Americans don't vote. Most of those who do vote have very little information and opinion on down ballot positions and propositions. If someone wanted to, I think they could persuade an indifferent or marginal neighbor or coworker to vote on a topic with an hour of face-to-face conversation. For context, the average american spends 4,000 hours a year consuming TV and digital media. Anecdotally, I too am part of the problem because I easily spend dozens of hours a year listening or complaining about politics, or reading articles to "stay informed".
>And then looking at it from another perspective: compare the change in well being of the average person in China versus the USA over the last 10 to 20 years - does the above explain the entirety of the differential? Or, might there be some other variables in play here?
This is a such a huge an interesting question, I'm not sure we could even define the scope or definitions on HN. I would be happy to follow up via email if you are interested.
This is all true, but my concern is that which seems to go unnoticed, and that is: the nature of "conversations at dinner tables and in lunchrooms", or in internet social media, forums, etc, seems to be highly influenced by the initial framing of the stories by politicians, journalists, thought leaders, etc. These topics do indeed get discussed, but are these discussions comprehensive of all dimensions and perspectives? Or, is it something more along the lines of: an event occurs, and various entities "get out in front of it", establishing specific narratives and perspectives for the public to argue over, and exerting ongoing maintenance of these narratives, discrediting or redirecting discussions that go off script, and amplifying those that are on script? If one pays very close attention to stories from a meta-perspective, I propose that these techniques can be seen clear as day (and this theory is far from novel- it's as old as the hills).
> The ideal parties to persuade are those with weakly held views. 40% of Americans don't vote. Most of those who do vote have very little information and opinion on down ballot positions and propositions. If someone wanted to, I think they could persuade an indifferent or marginal neighbor or coworker to vote on a topic with an hour of face-to-face conversation.
No doubt. And if executed skilfully, one may actually achieve the watered down proposals tabled by politicians, as opposed to the true democratic desires of the population (see: single-payer Medicare for All). And even when someone is on to this and tries to beat them at their own game?
"Alexandria Ocasio-Cortez rejects left-wing calls to force Pelosi to hold a 'Medicare for All' vote in exchange for her vote for the speaker"
https://www.businessinsider.com/aoc-rejects-left-wing-calls-...
Pay attention to the language and assertions in that article, as well as AOC's hypocrisy (prior complaints vs current stance):
"Gray and Dore pointed out that Ocasio-Cortez had previously suggested she supported holding a House vote on Medicare for All." "We can't even get a floor vote on Medicare for All," the congresswoman said in January. "Not even a floor vote that gets voted down. We can't even get a vote on it."
This is what I would call professional lying (or, Public Relations, Mass Perception Management, etc).
Is no one able to see the amount of persuasion that exists in "the news"? Am I imagining this? Do politicians not have a history of sticking to talking points?
Why does this abstract aspect of the situation so rarely get discussed on an intellectual site like HN? Why do we fight with each other over the same old object level nitpicks that one finds on Reddit? Are we not capable of more than that?
> This is a such a huge an interesting question, I'm not sure we could even define the scope or definitions on HN. I would be happy to follow up via email if you are interested.
I believe that this community should put some genuine honest effort into realizing that abstract perspectives and discussions on these matters is "the" path to eventual resolution, rather than succumbing to our intuitive Pavlovian behavior of reacting to the latest outrage ju dour that the media pushes into people brains. The underlying situations are not this dynamic, they are largely static across time...so, why do we not analyze and discuss them accordingly?
Is the way we undertake system analysis of Project Planet Earth (emotional, tribal, chaotic and disorganized, no documentation or tasks or timelines) similar to how we do the same for the complex (but far simpler) systems most of us work on at our days jobs?
EDIT: This "fresh daily news for the proletariat to argue about" approach to journalism reminds me of an old Joel Spolsky post:
https://www.joelonsoftware.com/2002/01/06/fire-and-motion/
>> When I was an Israeli paratrooper a general stopped by to give us a little speech about strategy. In infantry battles, he told us, there is only one strategy: Fire and Motion. You move towards the enemy while firing your weapon. The firing forces him to keep his head down so he can’t fire at you. (That’s what the soldiers mean when they shout “cover me.” It means, “fire at our enemy so he has to duck and can’t fire at me while I run across this street, here.” It works.) The motion allows you to conquer territory and get closer to your enemy, where your shots are much more likely to hit their target. If you’re not moving, the enemy gets to decide what happens, which is not a good thing. If you’re not firing, the enemy will fire at you, pinning you down.
>> I remembered this for a long time. I noticed how almost every kind of military strategy, from air force dogfights to large scale naval maneuvers, is based on the idea of Fire and Motion. It took me another fifteen years to realize that the principle of Fire and Motion is how you get things done in life. You have to move forward a little bit, every day. It doesn’t matter if your code is lame and buggy and nobody wants it. If you are moving forward, writing code and fixing bugs constantly, time is on your side. Watch out when your competition fires at you. Do they just want to force you to keep busy reacting to their volleys, so you can’t move forward?
>> Think of the history of data access strategies to come out of Microsoft. ODBC, RDO, DAO, ADO, OLEDB, now ADO.NET – All New! Are these technological imperatives? The result of an incompetent design group that needs to reinvent data access every goddamn year? (That’s probably it, actually.) But the end result is just cover fire. The competition has no choice but to spend all their time porting and keeping up, time that they can’t spend writing new features. Look closely at the software landscape. The companies that do well are the ones who rely least on big companies and don’t have to spend all their cycles catching up and reimplementing and fixing bugs that crop up only on Windows XP. The companies who stumble are the ones who spend too much time reading tea leaves to figure out the future direction of Microsoft.
Sound familiar? Why can we only achieve the smallest of accomplishments on matters that have (or would have, ex-propaganda) widespread public support, often for years if not decades, often bi-partisan? Might it be because the public is always reacting to the latest outrageous development, always in a state of confusion, never spending significant amounts of time focusing on one issue until it is fixed?
This problem space has barely been cracked open, and we're going to have to get big smart about it before it destroys us.
100% agree, but try getting any group of people to agree on this idea even in principle.
Most ideas become refined as they grow from very small open-minded spaces, towards mass appeal. So perhaps this idea can be refined as well... no sense in trying not trying.
2 Samuel 15:1-6
Now it came about after this that Absalom provided for himself a chariot and horses and fifty men as runners before him. Absalom used to rise early and stand beside the way to the gate; and when any man had a suit to come to the king for judgment, Absalom would call to him and say, “From what city are you?” And he would say, “Your servant is from one of the tribes of Israel.” Then Absalom would say to him, “See, your claims are good and right, but no man listens to you on the part of the king.” Moreover, Absalom would say, “Oh that one would appoint me judge in the land, then every man who has any suit or cause could come to me and I would give him justice.” And when a man came near to prostrate himself before him, he would put out his hand and take hold of him and kiss him. In this manner Absalom dealt with all Israel who came to the king for judgment; so Absalom stole away the hearts of the men of Israel.Just about everything we do has some component of persuasion or an appeal to biases. It's a huge can of worms.
Definitely not easy, but probably doable to a certain extent.
Clearly suppressing a truth is bad. But modern technology causes massive amplification of falsehoods in a way that just didn't exist in the past, and that's bad, too.
From a utilitarian perspective, fake news and persuasive technologies do real harm, and not just to the people who are persuaded. Unless something is done at the group level (i.e. by the state) the harm will continue. You might consider that an appropriate trade-off for liberty, but others can reasonably disagree without being weak.
The problem is that we are learning this is not true. Falsehoods can come so fast that the truth never becomes known. https://news.mit.edu/2018/study-twitter-false-news-travels-f...
A lot has changed since 1789, and there is no reason not to re-examine the assumptions that were made then in light of technological developments over the past two centuries plus.
It'd be like saying that all car driving should be banned because high speed driving is extremely risky. Of course that makes no sense, because there's a huge difference between driving 100mph in a neighborhood and just doing 25mph, and it's also the reason we have speed limits, which draw a line in the sand, past which we say "this is unacceptable."
The latter exists and has been used by Google to offer jobs to users for whom they may have an opening. (Or perhaps a competitor has openings and Google wants to ensure the best candidates not work there, which is what I would do if I were Google.)
That several gargantuan companies have and use this kind of insight into widescale human behavior is more terrible than "grandpa shared some nonsense on facebook."
Edit: clarification
Just today, in fact, I read this article: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3697726
Recent scholarship engaging with the impact of digital technology on contract law has suggested that practitioners and researchers will need to give proper consideration to the ‘role of equitable remedies in the context of contracts drafted in whole or part in executable code’. More generally, a raft of challenges stem from the increasingly active role digital technologies play in contractual relations.
Faced with these challenges, instinct may dictate attempting to tame the technological beast with a variety of regulatory responses spanning the full spectrum of possibilities, from legal requirements to voluntary codes of conduct or standards. While regulatory action may be a priority from a public policy perspective, the seeming trustworthiness of algorithms, and the consequent reliance placed on them by contracting parties carry the inherent risk of lack of autonomy and fully‐informed independent decision‐making that, in Australia at least, is addressed by equity through the doctrine of undue influence.
This article explores whether this traditional doctrine can adapt to operate alongside regulation in dealing with some of the challenges presented by algorithmic contracting. Specifically, it focuses on those contracts where algorithms play an active role not only in the execution, but in the formation of the contract, as these are the “algorithmic contracts” that challenge the very fundamentals of contract law.
Well obviously that's not what it means. What it means is that the speed of light is rather impossibly constant. Likewise, economics has this same issue. "Free Market" does not mean everyone has complete free will and has total immunity to persuasion. "Free Markets" defines a philosophy of trade where individuals can own and sell property. Whether or not people are 'easily affected by suggestion' as you put it, has got no impact on the concept of free markets.
Yes, it does. A free market is not just about individuals owning and trading property. A free market requires that all trades are voluntary. If people are forced, for example by law, to make trades they would not freely choose to make, you do not have a free market: forced trades are not voluntary. But if people are manipulated into making trades they would not freely choose to make if they knew the information the manipulator is hiding from them, those trades are not voluntary either.
If the marketing campaign is open, and I'm aware that that's what it is, and it doesn't make any actual false claims (puffery is another matter--that's basically unavoidable), then yes, it's my choice whether or not to be convinced.
If the "marketing campaign" is really efforts behind the scenes to present me with misleading information and to disguise the motives behind it, that's something different.
It is true that efforts of the latter sort are not new. What impact new "persuasive technologies" have on the frequency or success rate of such nefarious tactics is, I think, an open question.
There are some powerful benefits of free markets over command economies, such as decentralized self-organization, tendency towards the maximization of total surplus, and dynamic adaptation to changing conditions.
But those benefits are conditional on, for one thing, the rational behaviour of individuals.
There are ways to "hack" people such that they stop behaving rationally (with respect to the economy). The threat of violence or incarceration (maybe one you are equating with government intervention) is one of them.
But there are plenty of others: chemical addictions (e.g. cigarettes), systems that prey on the susceptibility of our dopamine-reward pathways (like slot machines, or your perhaps your facebook feed), etc.
Good policies in a free market economy are ones that make it more difficult to disrupt the beneficial aspects of a free market. I'd say good policy-making actually increases the free market-ness of markets in a messy, imperfect world.
Historically, our society has drawn a distinction between force (things like direct threats of violence or incarceration), which makes things done under such conditions involuntary, and things that are addictive but which people still voluntarily choose to do.
In some cases, such as smoking, we have ended up still imposing penalties on companies that purvey such products, but the basis for such penalties has not been the addictiveness of the thing, but the physical harms it causes, such as lung cancer. Addictive things that do not cause harms of the same sort, such as slot machines, have not been treated the same.
IMO that distinction is a good one: addiction does have a voluntary aspect that is not present in cases of simple threat by force. Treating addictions as though they simply override the agency of the person is not, IMO, a good idea. And I think a similar distinction can, and should, be drawn between simple threat by force and "persuasive technologies"; there is a voluntary aspect to the latter--people have to choose to believe what they are being told--just as there is with addiction.
Of course we also draw a distinction between acceptable persuasion and fraud or manipulation, which are unacceptable and punishable. That distinction is also a good one, and I think it can help to deal with "persuasive technologies" in a reasonable way.
All the economic theory of free market (and all the advantages of it) rely on a few core assumptions, like liberty to trade and set prices, but also lots of buyers, lots of sellers, full competition, low barriers of entry and full information.
Just as it's well known that a market that devolves into monopoly or oligopoly does not work like a free market anymore, or in the case of severely unequal bargaining power, the same applies in the case of information asymmetry, which also is well known to lead to a failure of free market.
So whether "people are easily affected by suggestion" does matter, because if that becomes the case and companies are widely using effective methods to do so, then the resulting economic structure of the competition is not like a free market.
People should affirmatively seek out sales materials, not be inadvertently subjected to them.
The new thing, and the key point here is scale.
You can discuss persuasion or cognitive bias away until everyone gets an aneurysm, sure. Some example: You can't not communicate. So even not persuading is persuading. What if the presidential candidate would just not give a scheduled speech? How do you communicate information objectively? Casual language creates bias, so does scientific language, simple language, passive language, active language. "Cognitive Bias" might as well be called "Cognition", since it is just how the brain works. You have to think "tree" immediately when you see one, even before you validated that all leaves are real and that the whole thing is not a projection on an transparent screen. Otherwise you can't function.
But: Big tech throws us in a situation where a small group of people influences our perception on a massive scale. Facebook changes a sentence on their homepage and a billion people read it. Youtube raises some parameter (yeah I know that's not how AI works) by 0.01 and the political opinion about the Grenfell tower disaster changes ever so slightly - for 30 million people. Google's filter has a tiny hole and some troll broadcasts wrong medical information about gout to 200k people.
Every time one of these things happen, the world shakes. Dozens die or survive. Demonstrations form and elections swing. Opportunities are wasted and ideas surface.
I am not arrogant to actually propose an easy solution to this, and I don't think there is one. Just be aware that "I can always go and stab someone" is not a good argument when you are discussing a fully automated drone swarm with kill authority.
If an advertising channel then shows ads that breach license they are liable. A fairly simple licensing process will come into play and we can find new ways to fund things
Edit: Yes i do get a lot of the issues around regulation of tech - it's almost like saying regulation of every day life which is really broad. And the different bodies and approaches will also need to be broad. But i am a believer in markets and individual decision making and i also believe that personal information has in the past few years become a genuine new ... commodity? And we need to raise that commodity into visibility - to be able to put prices on it openly. Maybe it won't work, maybe privacy is like a human right and can only be dealt with at that level - but i don't think so - privacy to me seems ephemeral and usually poorly defined. Longer discussion to be had
If you want to use Facebook without targeted advertising, you need to either convince them that they don't need all that money, or pay it yourself. And that's just Facebook.
In other words: the internet economy without targeted ads will be a very different place. Facebook will survive. imgurl / snopes / fivethirtyeight? Unlikely...
I suspect that this is a lot of the value of "the next billion". Why bring Facebook to people who have little disposable income? Because there are still ad buyers who will pay to reach those audiences if they have votes to give, or even just public support for some cause or another.
In fairness, there is an argument that at least some political advertising is consumption by politically engaged citizenry who don't stand to gain financially from the result; witness spending on unwinnable Senate races in the latest election, for example. But in general you have to assume that political advertising comes with some expectation of return, which comes out of the everyday citizen's pocket.
Joe's political campaign advertises to you because they want you to hate Bob, or be scared of those other guys, or whatever. This will help put Joe in charge, which is good for Joe and potentially bad for you, but it doesn't necessarily come out of your pocket. It comes out of the pockets of Joe and his friends. That might be taxpayer money, but you might not be the taxpayer paying for it. It might be private money from religious people who want gay marriage to be illegal. You pay for it with freedom, but not with money.
My point wasn't that there's a free lunch here. My point was that there are other mechanisms at play, costs of advertising that don't translate directly into cost of goods and services you buy.
Ever.
I have however seen plenty of ads trying to get me to spend money on something/someone.
And conflating the political process with the marketing process is absolutely pernicious in a democracy. The Brexit campaign used micro-targeting of individuals with knowingly dishonest ads that played to their biasses.
That's not persuasion - that's just manipulation.
Put simply, no one should ever be allowed to run organised campaigns based on lying and distortion for any reason. If you need to lie you don't have anything valid to say in the first place so removing you from the public discourse causes no harm to anyone.
But, really if we held politicians to honest, accurate and non-misleading statements ... well insert your own comment here
I agree that most of the time that's a bad thing, although it's possible it can be used by good people trying to convince you to be good, or raise awareness of an important cause. Probably much less frequently, but evaluating good and bad views is very subjective so I'm not going there.
Now perhaps all or advertising is manipulative , dishonest, and immoral. Doesn't detractfrom my point.
I wouldn't be surprised if it's much more than that. Most businesses sales follow the Pareto principle, 80% of their revenue comes from 20% of their customers.
I'm not sure how imgurl survives given the abuse workload; there is an inevitable death spiral of image hosting sites as they get cluttered by more and more adverts of worse and worse quality.
Stackexchange handles this very differently, too.
Imagine if we got rid of all advertising and what that would do to demand. I know the discussion here is centered on targeted ads, but I think there would be significant economic effects if their incidence was decreased.
The global consumer economy runs on all these extra desires created by ads, and ads do fulfil the function of informing customers.
If I have to see ads (and I really don't), I'm not against tracking ads per se, but the long-term storage of them. Given current technology and governance, there's not a way to separate these two.
In any case, there's significant economic effects to stopping tracking ads.
I hate this argument. It's tries to find a solution to sustain the status quo in the face of change. But that's illogical because if there is a change, then the status quo is going to change too.
The only way Facebook is going to stop doing targeted advertising is if they're forced to, likely due to new regulations. If that happens, Facebook's entire business model will collapse, so they'll either have to make massive fundamental changes to how they make money, or they'll die and be taken over by a competitor.
In either case, users don't "owe" Facebook anything. Consumers are the life blood of the economy, and thinking about how consumers should change their behaviors to sustain a corporation is backwards thinking.
It is that most users may prefer keeping FB and targeted adds, over losing FB and taking down targeted adds.
They sort of do. The money is converted into the utility of the product to each user. Technically, they could lower the utility of Facebook and pay you the rest.
Most users care about the enjoyment/utility/stimulation they get out of a social network. Some users care about their privacy. Needing an extra $30-60/year does not rank high on user priorities.
While facebook might make $200/US individual, their profit margin is only 30%.
Let's say every user gets 10% of the average, or $20/year. As you say, not a big priority. But some people would sign up, who otherwise would not. And some people would keep their accounts, who otherwise would not. Win for Facebook? Hard to say.
Now, what if Facebook gave each person 10% of what they personally brought in? That wouldn't take much more tracking than what they already do. But in this case, you'd have a lot of people maximizing their time on Facebook in order to increase their payout. Win for them, bigger win for Facebook. Heck, Facebook could even get rid of all their stupid dark patterns, and stop the endless research into more dark patterns, improving their bottom line even more. No need to trick people into engaging if they are voluntarily engaging for a small sum.
I still don't think the ROI for the user is there. The average FB user spends about 35 min per day, or 210hrs/year on the platform. Assuming 10% revenue sharing, that is $0.10/hour. If the average user were to double their time on the platform, they would make an extra $20/year.
This is a laughable incentive for most people, let alone your most valuable add targets (people with disposable income).
Maybe this essay should be forced to be presented on essays.com without attribution and compete on the ideas within rather than the implied authority of the author.
(I happen to agree with a lot of the content, but couldn’t completely compartmentalize that this was a persuasive essay against persuasion.)
-It's okay to use persuasive technology to push political orientations I agree with
-We want to defend democracy but we implicitly agree as commentators above the fray that people can't be trusted to make the right decisions and have to be manipulated towards our preferred orientation
But also, are there social media apps where you have to opt in to notifications? At least with native apps, I've only ever seen opt-out behavior, which doesn't really work with that analogy.
This is only funny because it's so close to the truth.
You know what makes it such a subjective question? The cigarette supplier sells a substance with an addictive chemical inside of it. Notifications are not a substance, the drug response is your own chemical (dopamine). Still an interesting question though.
Imagining this we can see that there is obviously some limit beyond which advertising must be curbed, and admitting this the question just becomes at what point must advertising actually be curbed?
Of course one could argue that such an advertisement must be for a product so wonderful and useful that everyone would want it - let us say immortality with youth and good looks - but if that were the case such a product should be recommended enough by word of mouth and the evidence of all the old people becoming young and good looking when taking it.
But I think the real question that needs asking here Winston, is when are you going to just give up and love Big Brother?
I have wondered if in the future, movies/TV shows will be personalized with your name. I have found that when I watch a movie where the main character shares my name (and therefore other characters say the name a lot when talking to/about him), it makes the experience more immersive. It would seem fairly trivial to substitute in other names in a pretty smooth way (it would get tougher if you had to also adapt things like a business card that is visible on-screen, however).
Of course, this could be taken to the next level if you changed the faces of the actor via deepfake-like technology. I don't know how actors would feel about this sort of thing, but hey maybe it would open up the door for a bunch of new actors, who would essentially be a blank slate for customized faces. Imagine a world where Hollywood actors don't have to be good-looking!
That’s has to be so weird to hear your name in shows/movies.
I have noticed, for example, that anglosphere seems to adopt US Hollywood/cultural tropes much easier and faster than other countries. Down to, for example, people using the word 'like' every other word or speaking in a certain tone of voice being quite pervasive in England but to my understanding relatively rarely seen in, say, the Netherlands.
I've been long hypothesizing that my English "self" is running in a sort of a "virtual machine" where I can step back, examine and pick apart the input for the parts that are relevant or beneficial, then continue processing.
Can we maybe go one step back in the discussion and not only discuss what we should do about it, but simply ask does it even work?
There's Zuboff's book about surveillance capitalism that echoes much of what the blog post talks about, that recent netflix documentary that everyone was talking bout and so on, but how much evidence is there that this isn't all just mostly bullshit?
When the Cambridge Analytica scandal broke they used the buzzword 'psychographic targeting'. turns out, psychographic targeting doesn't even really work[1]. A relative recently sent me an article about China allegedly using mind-control helmets to control their thoughts of children attached with a picture of children wearing helmets with blinking lights. I have yet to see a facial emotion detection system that labels Harold[2] of the "Hide The Pain" meme fame as anything other than 'happy'.
I'm more afraid of how bogus all these systems are and the unquestioning power people attribute to tech, which itself enables these firms. It's no wonder they keep inviting Yuval Noah Harari for talks, they must feel flattered.
[1]https://www.nature.com/articles/d41586-018-03880-4
[2]https://static.independent.co.uk/s3fs-public/thumbnails/imag...
It stands to reason that we need to draw lines around what appropriate persuasion looks like not because these new techniques are definitely abusing our freedoms but because they might and in theory could. The potential in itself should be enough to treat the role of technology seriously and consider what the boundaries should be.
I think this is why Silicon Valley loves to invite their critics who argue along these lines. This criticism doesn't deflect their own notion of supremacy, it just makes them look like bond villains, and they secretly love it.
Showing that in many cases the emperor has no clothes, that complex human behaviour can't be reduced to some ML and that in many ways it's embarrassing self-aggrandizing marketing is I think, the much better way to reign in these firms.
Whether or not the influence of technology is overblown, the fact still stands that we need to draw boundaries around how technology can be used for persuasion. The question should be "where is the line?" not "should we even bother drawing one?".
For example, regression models aren't exactly ai magic, but unchecked, they compute discriminatory insurance premiums in a way we find morally unacceptable.
The naked emperor still needs keeping in check.
In reality, the voting public ultimately has 100% control. Most of the people who spend time lamenting about the power of political advertising, lobbying, ect have never once knocked on a door and tried to change someone's mind about an issue.
Unfortunately, believing in a world view where the you are disenfranchised victim is incredibly convenient. It allows individuals to absolve themselves of responsibility while doing nothing.
It makes sense that empowered citizens in a society would naturally want to protect their rights to not have their natural human biases gamed or manipulated. Thinking ourselves above human nature and beyond the tactics of persuasion would be naïve but we needn't see ourselves as helpless victims. Instead, we should act to defend our rights and remain empowered and in control. We're not disenfranchised and we can introduce regulation to ensure that our freedom is preserved.
They say sufficiently advanced technology is indistinguishable from magic, but in the day to day business world it manifests more like "Technology I haven't bothered to actually understand is indistinguishable from magic". And everybody loves magic!
https://news.ycombinator.com/item?id=25415052
Is an example of a self-fullfilling prophecy. Bad words aren't used because they are bad. They are not used to game the system and hence become bad words....
...Sorry, English is not my native tongue.
The industry described here is heavily degraded and removed from reality, it is a sorry mix of gambling, lying and posing. It was predictable they would swallow AI systems for their "work" like their lives depend on it. This is blind faith, you could use a prophet to do the same.
There would be no net negative effect if it vanished overnight aside from slower price building. Regulation could do so much here compared to rules for advertisement or AI itself.
This is actually a topic about bias, since people use tech for self-validation to a large degree.
Anyhow, if you go look at the Wikipedia entry for "Neurolinguistic Programming" you'll see that it's coated in warnings that it's a pseudo-science. So that's where we stand today in re: state-of-the art persuasion technology. Most folks have never heard of NLP, many that have are openly skeptical (verging on hostile) and some people have entered into the Information Revolution proper. (Meaning they know and use this body of knowledge.)
Given that we have allowed software to fall under patents, and that this knowledge constitutes the software of the mind, I'm reluctant to have it "go mainstream". I'd hate to imagine patent wars over IP that is essentially just structured thought...
On the other hand, ignorance of the "operating system" of the mind causes unspeakable suffering. (I myself was cured of severe depression, just to add a personal, anecdotal, note.) And the differential between folks who know and folks who don't is also problematical. That would seem to argue in favor of rapid and widespread dissemination.
Then there's the problem of the self-referential nature of persuasion and the limits thereof: can you limit persuasion if I can persuade you not to? Either persuasion tech doesn't work and so the laws are unnecessary, or it does work and can be used to affect its own regulation.
That being said, its seems the event that has brought the concern over persuasive technologies to the fore is the election of Donald Trump. New sorts of persuasive technologies may have Trump over the finish line, but I'm pretty convinced that he ran a competitive campaign without them. He identified policies and cultural grievances (immigration, anti-elite sentiment, trade, and white identity politics for example ) that no one else was talking about, ran with them. I dislike that these subjects resonated with enough of my fellow Americans to win an election, but they did. To retreat into a mindset of "those rubes must have been tricked" is to deny them agency and is problematic in and of itself.
We live in the attention economy. We have medias monopolizing that attention.
It's the Supermarket of Ideas™, right? Maybe the Freedom Speeches™ and Freedom Markets™ camp followers could advocate for some competition.
I dunno, maybe something crazy, like a doctrine of giving interventions their fair share of oxygen. For example: After Alex Jones spins up the peanut gallery, trained psychiatrists can talk them all off the ledge. Make sure they all get a cookie and some nap time.
Call it the Milk & Cookies Fairness Doctrine.
I thought about researching how humans react to persuasive technology and start self regulating by installing adblocker/deleting apps...
I am all ears if there are other interesting questions you might think of :)
- what is the range of ads to be served like? I rmeber thebfounder of duck duck go saying lack of PII made little difference to him - someone is searching for "PC Monitor" then the range of ads seems pretty obvious. Is this true in "persuasive" mode? Are there bidders trying to serve political ads or lingerie when I search for a monitor? How can you tell?
- is this a zero sum arms race ? A study I cannot find should that Political races are decided by who the public prefers not by total spend. Congressional races showed that if ad spend was matched the winner stayed consistent - it's only when one side out spent the other did ads have an effect. If I am outraged by ad X do ai also get served the anti-X advert? Would it work if I was? (this is the equalicalent of how many adverts saying fruit and vegetables are good for you outweighs adverts for sugary drinks? There is probably a lot of metrics out there on that - how comparable is it?)
good luck
I am personally intrigued by the idea that people, who are aware of persuasive tech, PII, captology and marketing efforts skyrocketing in the attention economoy, start to mistrust products/ services or political agendas that spread their message with the help of such tech. Might only be my bubble though...
Pedantic, but there's only limits on how much you can buy while clearly drunk. Although, I suppose passing out is sort of a limit on how much you can drink.
> Emotional appeals have likewise long been a facet of political campaigns. In the 1860 US presidential election, Southern politicians and newspaper editors spread fears of what a "Black Republican" win would mean, painting horrific pictures of what the emancipation of slaves would do to the country. In the 2020 US presidential election, modern-day Republicans used Cuban Americans’ fears of socialism in ads on Spanish-language radio and messaging on social media. Because of the emotions involved, many voters believed the campaigns enough to let them influence their decisions.
... where he sets up (anchors) a reprehensible example of an emotional appeal (19th century racial bigotry) to reflect a 'reprehensible' example of modern-day emotional appeal (animosity towards a social/economic philosophy.) To reinforce his point he purposefully neglects to mention it was Southern Democrats who spread fear against 'Black Republicans' and instead uses 'Republican' twice (as a callback) to tweak the more modern-day version of that political party.
The irony of Schneier using his long-standing and influential blog to insist we 'have a serious conversation about limiting the technologies of persuasion' is not lost on this reader.
Edit--Feel free to downvote, but perhaps explain where my comment is wrong?
It will be hard to put guardrails on this stuff. It's pretty much 100% "gray area," easy to cast in many ways.
By all accounts, the election system in the US is working, most people aren't antivaxxers, and belief in an undetectable wizard in the sky seems to be falling. We are winning the war against misinformation.
Why are so many smart people claiming that we aren't, and that more censorship is required?
> [TFA, Schneier] In this regard, the United States, already extremely polarized, sits on a precipice.
Citation needed.
https://www.scientificamerican.com/article/why-social-media-...
I think people place far too much stock in US partisan screaming. It simply doesn't matter that much.
I'm not arguing that the US isn't polarized, that much is clear. I just don't see any precipice.
Social media is just a new medium; but an effective one that allows scale. I'm old enough to remember when television occupied the same slot.
These types of campaigns are usually multi-pronged, sustained and coordinated. They aren't really effective, unless they are all managed by a central director.
Dog whistles only work if you train the dogs.
I think that it's about where the effort is directed. The US is a huge target. This nation has an oversize influence on the world, and it's natural for other nations that either resent this influence, and/or want to exert their own, to do as much as possible to damage the US.
Looks like they've done a fairly good job.
Witchcraft accusations (which honestly in the 80's there were a lot) are way down. People understand glass is not a liquid. All music is now mainstream. String Theory is being exposed as a religion.
But people are very polarised on what they do disagree with. You wouldn't fire someone for being in the KKK 30 years ago. Now you fire someone for naive accidents.
I don't know what to think on your point.
Over time things should stabilize as we get social antibodies against new forms of persuasion, and a new balance of power between groups that organize differently because of new communication patterns.
For us on HN, Internet may be something that's existed for many decades, but social media and smartphones really changed things for the masses starting around 2008-2010 or so.
I'm actively contributing to a tool called Pol.is[3]. It's designed to be consensus-building tech. It's part of some pretty rad democratic processes in Taiwan[1]. It basically just puts a more approachable visualization over some overwise intimidating (but basic) statistical methods of "dimensional reduction", like PCA or UMAP. This visualization lets any participant explore all the statements that each statistical group agrees and disagrees on together.
These "opinion groups" are just neutral statistical clusters. That feels important. It's just an integration of gut-feel statements that people perform a hot-take of: agree/disagree/pass. But it turns into this machine for everyone to independantly uncover the deep stories of each group. Once groups form, for complex but focussed issues, there are often 3-4 groups, even if you maybe thought it was just "us and them".
Light-hearted example: https://pol.is/3ntrtcehas
This tool does lots of interesting things psychologically.
One of my working theories of what makes it unique (I admit to having a few) is that it incentivizes the most passionate participants in a discussion to do the hard work of finding ever-more-nuanced "majority opinions" that everyone agree on under the differences. (These are elevated in the UI.) The nature of this incentive is up to the people who are designing the process in which the tool is embedded. For e.g., they might say that every "majority opinion" uncovered will shape an agenda item topic in a future livestreamed meeting of powerful people. So now you have the most passionate participants (who might otherwise shake apart concensus if their energy were directed otherwise), trying to craft clever statements to "manipulate" people in other groups to agree. But in essence, what this means is that those passionate people are exploring the consensus statements of opinion groups other than their own. They want to understand the group so that they can drop a statement into the opinion space between groups. But even if the intentions are not noble, and someone is just trying to "trick" others into agreeing, that participant is still doing deep work to delve into the theory of mind of the other groups, as evidenced through that groups consensus statements. Basically, the most active, passionate and potentially disruptive participants are encouraged to learn about their opponents in order to sway them, and in doing so, they accidentally indoctrinate themselves into empathic bridge positions. And these bridge positions are perhaps very important to stabilizing consensus and resolving conflict[2].
I'll admit that I wonder what a political or democratic system might look like if we used tools like this to elevate the voices of participants who straddle bridge positions between groups. Like what if we had democratic processes where instead of sending ambassadors from the center of opinion groups, we actually elevated those from the boundary spaces. (Maybe this was always the benefit of systems that use random sortition to elect leaders?) The selection of those people would be relatively politically neutral, as it's not a subjective choice, but a statistical one. Yes, one can probably "fake" being moderate, but to truly embody it, perhaps that changes you. And what effect would it have to send these bridge people (even transiently, for just one topic) into a position of power, to pass judgement?
[1]: https://www.wired.co.uk/article/taiwan-democracy-social-medi...
[2]: https://www.ted.com/talks/mariano_sigman_and_dan_ariely_how_...
As I see it, the problem statement is as follows:
1) individuals are subjected to vast amounts of political persuasion in their daily lives. This persuasion can be purchased by the highest bidder to subtly influence public opinion in a way that is counter to it's own interest. Ultimately, this can weaken the marketplace of ideas, rational discourse, and consensus.
My idea is an app, forum, or social group which pairs individuals to discuss issues in a manner that builds consensus. Users would identify their position on issues as well as how strongly they hold that position. They would then be offered the opportunity to discuss and issue which they care deeply about with someone who is indifferent.
This could yield a high persuasion return for time invested, and help build consensus. This could have an effect similar to door to door campaigning, but with the ability to target politically active individuals who a with high impact topics. One thing that I find particularly attractive, is that it would focus the discussion on individual issues, opposed to platforms or parties.
What do you think?
> This persuasion can be purchased by the highest bidder
Yes. In other words, they're purchasing weighted edges in the social graph: buying new edges (that send peer signals toward the target within a graph of social influence), or amplifying weight of existing edges representing peer relations. The whole attention economy is not actually about the knowledge or content that flows through this network, rather it's all about buying/building/fortifying edges/connections between marks and those who will exert influence on them, in the direction favoured by the buyer. It's the shape of the network that's damaged, not the content flowing through it (which we've never had more access to). Empathy is just an existing biological solution to a similar network problem, maybe even a biological technology (sorry, can't help it with the weird jargon here -- was a biochemist in a past life).
> My idea is an app, forum, or social group which pairs individuals to discuss issues
It's a neat idea, but to be clear: why would people participate? I feel that without a self-interested reason, audience would bias toward process-oriented and highly empathic people who already care for the sake of caring. (I would participate, for instance.) imho some aspect of participation needs to give the process and participant teeth they didn't previously have, or give the participant power in a larger system. That's why people vote. That's why they run for election. That's why they join boards/committees. My mindset is interested in empathic tools primarily where they marry power-seeking and empathy together again in a new process.
It's a neat idea though, and don't take my words for discouragement pls! It's one thing to seek social/tech tools that sit beside charity or art or altruism in the cultural landscape (which it sounds like you're rolling around), but it's another to seek social/tech tools that sit alongside elections or Robert's Rules or other governance processes (where I believe Polis operates). We definitely need more people looking for opportunities here, so I'm grateful you're thinking on these things :)
>why would people participate? I feel that without a self-interested reason, audience would bias toward process-oriented and highly empathic people who already care for the sake of caring.
My thought is that the "selfish" incentive would be to make impact on an issue you care about. In exchange, you give someone a chance to talk to you about an issue they care deeply about you don't.
The underlying premise is that caring and reasonable people can and do have differences of opinions. The "teeth" is giving access to likely voter will listen and not have a dogmatic position. I for example, would spend an hour to double the my effective voting power.
I have little interest in running for an office, and don't think that politicians are particularly effective about changing the minds of voters. They are tied to a multi-issue platform which is incredibly cumbersome, and disinterests many individuals from the process overall.
I think the FDA was approving medicines long before the precise molecular mechanisms were known. Even with unknowns you can still study outcomes and legislate around them. Instead you seem to propose "oh we don't know enough, so we should just let it run wild".
I do agree that sensible legislation around this is hard to impossible to do. I'd rather see some more research into the effects funded before jumping directly into legislation.
My point was to emphasize how little we know about psychology and how little good a law that addresses our current technology will do in the long run, since the psychology-focused technology we have 50 or 100 years from now will likely be vastly more dangerous.
The image I have is of a caveman who, upon seeing one of his tribesmen kill another with a rock, proposes to regulate the use of rocks. Good idea maybe, but give it time and rocks will be the least of his troubles.
However it's unfair of me to dismiss our real life solutions to real life problems by pointing to a problem that might come to exist in an imaginary long-term future. We can only make the best with what we have.
Maybe I need to update my mental model of the law from an eternal-and-perfect view to an incremental-and-imperfect view, especially when it comes to the intersection of law and technology.
What do you imagine is going to be the horrifying fallout of telling businesses and politicians they're not allowed to build and act on detailed profiles of people?