Manipulating Human Psychology To Turn Users Into Addicts

We are in a society that has the distractions of Huxleys world with the surveliance of Orwell. The only thing different is the fact that no one expected our computers to get so small . That we would stare at screens that watch us every hour of every day. The anime psycho pass is much more relevant .

Orwell wasn't criticizing technology. "Animal Farm" is not about animals or farms and "1984" is not about TVs with cameras. He took the lust for power and turned it to eleven (hence the dystopian setting) to write a book about oppression.

"If you want a picture of the future, imagine a boot stamping on a human face — forever."

Have you read Huxley's sequel to Brave New World? It's called "The Island." In it all of the technologies from BNW are used to set people free; to enhance the human condition.

I stand with Nir (and Huxley). These technologies are powerful, but that doesn't make them evil. They can be used for good.

I'll go with: d) All of the above.

Speaking of Hollywood, did someone notice that nearly every Hollywood movie made the 50s contained at least one scene in which the male star forces the female star to kiss him until she surrenders? At that time it was considered an obligatory part of the plot, now for such a thing you could go to jail. Wasn't that psychology manipulation as well? Or is it now? Or both?

Seems like you've read 'Amusing ourselves to death' by Neil Postman: https://en.wikipedia.org/wiki/Amusing_Ourselves_to_Death

>I find it dumb, frankly.

I find your comment rather ignorant. It's entirely possible that the scenarios can in some way coexist. I'd also argue that for 2001 and The Matrix, technological danger is not the main subject.

>Then there is the political criticism along the lines of George Orwell and some Luddites: technology is a tool for control and domination either by governments or evil capitalists.

I would not describe it like this. Proponents of technology but critics of the rationality it has created exist, most notably Marcuse. He criticises the development both in the capitaliat countries abd the USSR from a Marxist perspective in One-Dimensional Man, a widely acclaimed analysis of the rationality of creating false needs. Please read it, you may like it given your viewpoint. He very well integrates the last paragraph of your comment. Man is one-dimensional due to the rise of technological rationality in all expression, sexual, artistic and leisure. Nothing remains undominated, not even the great unknowns of nature and the human mind. Everything is integrated and rational through paradoxical irrationality of overproduction and false manufactured needs through advertisement.

Remember the startup mantra I've heard pg and others state: make stuff people want.

This is true locally, but globally and taken to the extreme it's a nightmare. People don't know what they want, and if you were to give them everything they wanted, you would destroy them. Take a look at anybody who's suddenly come into a lot of fame and fortune. The results are usually appalling.

> [...] technology is a tool for control and domination either by governments or evil capitalists [...]

> [...] technology will entertain us to stupidity, apathy and alienation [...]

Perhaps a combination of both. The whole concept is as old as the Roman empire, [1].

[1] https://en.wikipedia.org/wiki/Bread_and_circuses

Huxley vs Orwell:

http://www.anorak.co.uk/wp-content/uploads/2013/06/huxley-or...

You might also find the work of Marshall McLuhan to be interesting:

>Instead of tending towards a vast Alexandrian library the world has become a computer, an electronic brain, exactly as an infantile piece of science fiction. And as our senses have gone outside us, Big Brother goes inside. So, unless aware of this dynamic, we shall at once move into a phase of panic terrors, exactly befitting a small world of tribal drums, total interdependence, and superimposed co-existence.

-The Gutenberg Galaxy (1962)

You might want to check out some of the criticism advanced by people such as Tristan Harris on the tension between the user behaviors that tech companies optimize their product design, and the goals users hope to accomplish through their use. I do in some ways support Nir in that we should learn how to harness these types of technologies for our own good, to 'supercharge' our ability to accomplish what we want, but I do not trust any company dependent on ad-revenue (or backed by Wall Street in general) to help with that.

I'd also recommend checking out The Attention Merchants by Tim Wu and Addiction by Design by Natasha Schull for background on the history of the commodification of attention and of the

"Any potentially addictive software/service should offer a way for users to lock themselves out of their own account for a predetermined set of hours."

It's really noticeable that Hacker News provides exactly this feature, and absolutely no other site that I've seen does.

You might like an app I made called Space.

The idea of a hard block on tech is intuitive, but it doesn't do anything to undermine the habitual learning in tech addiction; it just frustrates it for a fixed amount of time.

Space focuses on slowly undermining the compulsions behind tech addiction. So that, over time, you don't have the impulse to open the app anymore.

If you try it out, let me know what you think!

http://youjustneedspace.com/

I agree with you, but I think it's more than just greed. These companies consist of many employees. Depending on how you look at it, all those employees are contributing to the problem indirectly by working for the company. So one question that comes to mind is are these employees greedy too? What about the interns that get minimum wage? Are they not simply trying to make a living?

So I think the first part of your comment is more accurate in the sense that people rationalize things.

In that regard, I think we should look at schools emphasizing values in terms of work ethics. When I was in college I took a business ethics class. I realized that was the first time in my entire education that I was learning things about anything even remotely related to this subject, not to mention, the class was optional. That's a sad thing to happen in only the last years before entering the permanent work force.

I think there is a great truth to what you said, but the reduction of Tech to just media hides a set of critical and novel dangers in doing so.

Tech provides a new set of tools - both finely grained, invasive, and mass produce-able - which target areas of human existent earlier protected just because of how arcane and nebulous they were.

As human beings, we have many tendencies that are unguarded for buffer overflows, or malicious code injections.

With these tools, and corporate/shareholder incentives - firms and teams will run amok.

Its no longer just 1 way media, its now very close to mind invading media.

Sadly the only solution seems to be to ditch the smart phone whole.

We don't need "increased awareness", we need a social decision system (aka "an economy") that rewards positive social outcomes and disincentivises negative social outcomes.

At the moment we have the opposite. No one cares how you make your first billion or two, or how people suffer as a side effect. They only care that you have a billion or two. Toxic side-effects remain invisible and uncounted.

There's a certain irony in attacking EA for building a game economy that's closer to the effort/reward profile of the real economy - for both players and owners - than any other game economy to date.

I use Facebook mostly to keep up with a few old friends. It saddens me to see what I can only call radicalization happening to many of these folks.

There is no better.

From many angles there are too many things working to passively shore this edifice up.

The best solution I found in years was after randomly catching up with a good friend. He just gave up on the smart phone.

Hes not a Techie, he used to love video games - but he has a unique quality to be embody Human-ness. Hes dumped social networks, and dropped smart phones.

In short the way out for those who can manage it is cold turkey.

I now suspect that the people who can handle this, are not on HN or social networks anymore

Ironically, clapping, is the very device Medium uses to keep writers writing, the same stimuli that releases dopamine in other platforms.

And yet, the author decided to write his article on the very site that uses techniques Nir talks about to promote engagement.

Yes they are completely morally bankrupt

They invented a whole language to paper over the fact that their entire field is weaponized psychology employed against their customers. Do a noun for noun substitution to unpack a marketing meeting, and you'll want to puke.

Frankly, it's shameful for any company that says they care about their customers to employ marketing psychology tactics. Marketing and advertising are (in their modern, virulent form) simply an attempt to hack viewers to shift the locus of control from the customer to the business. (Other objectionable things about tech have this kind of behavior, too.)

Frankly, it's just disgusting and I can't blame customers for really viewing companies negatively based on their advertising practices.

(If all that wasn't bad enough, online advertising often comes with literal malware ontop.)

Also, I would like to point out three things to folks who are asking "What about personal responsibility? What about agency?" here and on the article comments.

Facebook's shadow profiles take away any 'agency' and 'personal responsibility' out of the equation. While I don't yet know Facebook using shadow profiles to make someone's life worse off, their penchant for saying "Screw it" to any ethical considerations makes me feel its only a matter of time before even that crutch goes away.

Secondly, there is the complexity. You cannot take personal responsibility for something that you don't fully comprehend. At the same time, we are at a point where AI produces data insights that even the AI builders cannot reason about. For example, how does someone take responsibility for clicking the Like button on a post and then finding out that most of the people who liked the post were (say) closet anarchists which was then correlated with them being the best targets for some kind of subversive advertising from (say) the NRA?

Thirdly, there is the asymmetry involved in "moderate use". It is possible for Facebook to create "automated notifications" implemented at a very tiny cost to them in terms of their resources/profits while the effort needed on your part to make sure you don't turn into a Pavlovian "notifications" dog comes at an extremely high time cost to you.

I used to think that, until Sean Parker recently stated it was a conscious, deliberate, decision to exploit people.

actually maybe the article just needs an editor or at least a proofreader.

It's not too far. Maybe You don't feel effects of antisocial media, but i do. Few years ago if friends wanted to go swimming. They were calling each other, there was human interaction. Now it's event on facebook. If You dont have favebook you're basically not a friend anymore, because noone is going to call You and invite. This is the effect of social media on our civilisation. It's like in '60 doctors were recommending smoking as stress reliever and everyone agreed until they knew better.

http://fortune.com/2016/06/22/loneliness-is-a-modern-day-epi...

https://www.cbsnews.com/news/many-americans-are-lonely-surve...

I wouldn’t call stackoverflow addictive - if anything I use it too little! - but I do have that problem with some of the linked network sites on stackexchange. “Why yes, who would win in a fight between Aquaman and The Flash is more interesting than why the third argument of the ExampleFooAPI constructor doesn’t behave itself!”

(I’m glad adblock lets me hide that block. Really helped me avoid those distractions).

The entire post is addressed to him, and not in a positive way. I tend to agree...

If my writing makes it less likely that some would-be Facebook competitor can use mass-scale psychological manipulation to get an edge, I can live with that.

Is that necessarily a bad thing? I feel like social media has destroyed an enormous amount of value by making it harder to have authentic relationships.

I highly doubt a blog post will have that effect.

The new Facebook may be kinder and gentler and not designed to exploit you.

The killer feature may be that it does not exploit your human flaws.

People are always surprised what new needs we have for technology. Now we need software and services that respect us as humans.

In other news, it's really hard for the "next British East India Company" to enter the Chinese Opium market

Drug dealers don't have intricate control over the effects of the drugs they're selling. I imagine if synthetic drugs were legal in general rather than specific compounds you might see similar problems pop up (e.g. 3-fluoroethamphetamine, a compound screened in rats to be maximally addictive) but it's reasonable to advocate for a drug market that avoids this.

I think Agency is always understood to be a part of the equation - and the people with the agency are not the issue.

By analogy - If the population of a city was exposed intentionally, to a cold virus - it is not the people with natural resistance, we are concerned about.

We are glad those people are not affected, and know they are not part of the % affected by the virus - and then we focus on the people affected by it.

---

If I were to engage with your point in the spirit it is intended, there are several issues -

1) Tech recruits the best of the best. IT pays for the smartest people in the world (and this is not hyperbole), to get past agency. This is not hard core drug dealer style sale of addiction. This is candy crush, happy smiley imagery, "just one more game", style behavioral addiction.

Most people don't even know they are being addicted/affected

2) This is all at internet scale. Just on a probabilistic level, the casualties/infection is going to vastly outperform personal agency.

3) The network effects of having more people doing this, at this scale, pushes it to crisis levels very quickly. It wouldn't matter what the few immune or capable can do.

People are amoral, acts are immoral. An act by an amoral person can certainly be immoral.

Manipulation had always been there but what's new is that the best minds of a generation are working on exploiting human weaknesses with the help of technology to a new level.

Same for surveillance. It was always there but now it can be done on an absolutely unprecedented level.

In standard / common usage, selling and marketing an umbrella to someone is not "manipulation".

Your definition of "manipulate" is not what most people mean in these contexts. Your use of "manipulate" in such a way muddies the waters and distracts from the core issue.

Putting word choice aside, the core issue here is _not_ about people's preferences (e.g. many people don't like to get rained on). Rather, the core issue is about unknown and unwelcome influences, particularly influences that operate on parts of our minds that are hard to combat due to human nature.

Facebook's manipulation of our attention is conceptually related to how casinos trick people into staying longer than they would otherwise.

You're missing the static vs dynamic distinction. Facebook can guess with eerie accuracy whether you're about to start a relationship with someone, whether you recently got pregnant, whether you're going through a depressive/manic phase...the list is endless. Facebook changes itself by taking advantage of this information. An umbrella can't do this.

The point though is that companies act to maximize 'user engagement', a goal which may be at odds with what users actually want to get out of a product or service. Whatever they can do to get just 5% more views or clicks or time on sight is 5% more ad revenue for their investors. Nevermind if that 5% comes from time I'd rather been spending elsewhere and will end up regretting. They incorporate design choices that cause this, whether by autoplaying the next video, implementing a continuous feed, all while controlling exactly how much content they need to show before you finally get sick of it. They are stealing the surplus the promised you.

And suppose you are of psychological steel. Should those who aren't still be exploited? There are entire industries and schools of design whose goal it is to extract as much money as possible from users, take slot machine design for example. (I'd definitely recommend looking up the work of Natasha Schull). Do you consider that practice immoral? Nir's existence is a testament to the fact that product designers use similar techniques

You haven't been manipulated into buying an umbrella at all, because deciding you'd rather stay dry than wet is a straightforward choice; a product that offers you that choice is a straightforward product which provides a tangible service.

To be manipulated, you'd have to exposed to a constant stream of ads telling you how wonderful umbrellas are and what an unattractive loser you are for not owning one; how desirable people with high status use exclusive umbrellas while most people get by under a wet newspaper; and how you should buy a new umbrella every year because they're constantly improving, and this year's umbrella already has more a billion users worldwide and offers 23% more anti-rain features thanks to the exclusive Umbrella Corporation[tm] lifestyle eco-system. (And now here's a video of attractive people smiling, and of a world-famous umbrella designer talking earnestly about engineering, design, and craftsmanship.)

You'd also see paid testimonials and likes on social media sites. They'd appear to be spontaneous and sincere praise, but in fact they'd be factory farmed and produced to order by unhappy people working in open plan offices paid so little they can barely afford the rent.

And you'd have an umbrella that only worked reliably some of the time, so you'd have to keep considering the purchase of an improved model - in the vain hope you could finally end your constant nagging frustration with umbrella technology.

Worst of all, you'd believe you were making a straightforward umbrella-buying decision entirely on your own initiative, using perfectly objective and rational criteria.

> I've been manipulated into liking broadband because it provides the internet faster than dialup.

You weren't manipulated. Manipulating you into buying broadband would entail creating the artificial perception that broadband is faster. If a company produces a product that performs as it is supposed to, and it solves a problem you have that was not artificially created, then you're not being manipulated. Same for umbrellas. Not the same for cars: the auto industry has pretty clearly lobbied for policies that ensure sustained demand for cars as opposed to public transit.