Hacker News Reader

Comment by ksdale | Hacker News Reader

I agree that it's unlikely that anyone can predict broad, long-term consequences of any particular technology. On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. It's not a huge leap to go from "I'm pretty sure that certain people will become obsessed with this," to "It's probably harmful for lots of people to become obsessed with this."

If a person can predict the consequences of design decisions well enough to get paid for it, they'll probably have some idea of potential negative consequences as well.

View on HN

There may be cases where negative consequences can be predicted, but this sure isn't it. Of all the predictions I heard about Facebook when it was new, "fake news" and "depression from comparing yourself to friends out having fun" and "an increasingly clickbaity and outrage-stoking newsmedia" were not among them. Not even close. Facebook is a great example of how programmers are not good at predicting negative consequences. The predictions I heard were mostly based on comparisons with Myspace and Livejournal. We are not good at this. Plumbers shouldn't try to do your electrical wiring, doctors shouldn't try to manage your database, legislators shouldn't try to pick your programming language, and programmers should not try to determine how society will react to technology.

I think you (and some other commenters here) are making the mistake of saying that because we cannot predict the future perfectly, that we are absolved of any responsibility for thinking about negative outcomes.

> programmers should not try to determine how society will react to technology

That is exactly one of the most important parts of our job.

Do you think doctors should not think about the wider impact of treatment decisions? Legislators should not think about the second-order effects of the laws they pass?

Yes, prediction is difficult (especially about the future, as Yogi Berra said). We all know this. Most of the things that most programmers are doing have known and predictable effects, because most of what we are doing is not new. If we had no idea how society would react to technology, why would we even develop it? Of course thinking about the effect that your work has on the world is a basic responsibility of any human being, how it could possibly be otherwise?

Respect.

I'm not sure it's so much a matter of picking negative consequences out of the blue sky of possible consequences, but rather wondering, "What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?"

I agree completely that it's hard or impossible to just predict in general how society will react to technology. But in many cases, there are people designing systems specifically for the purpose of getting people to behave a certain way. This implicitly acknowledges that they believe that they can affect behavior in a predictable way. If they can make predictions one way and be correct enough to profit from it, they should be capable of making predictions about things that could go wrong.

I agree completely that predicting something like what Facebook will look like 10 years in the future is impossible, and there's surely a lot of research where the possible consequences are infinite. A lot of results in CS research are like someone developing a new material - "Like, sure, this could be used to make a new, extra deadly tank or something, but it could be used for literally anything." In that sense, it's not useful to make predictions, but I think there's also a lot of research going on at Facebook and Google that looks closer to what I mentioned above, where they absolutely have an intention to change people's behavior and so probably also have the ability to predict the consequences of their complete success.

These are two different set of people: computer scientists develop algos / ml / etc, and want to publish them. UX designers , entrepreneurs, and other customer-focused people in the industry observe human behavior, and (amongst other activities) design traps around them.

It is not useful to expect good predictions from the former group, and the later group isn't really interested in publishing in CS.

The Challenger disaster is a good example of the engineer vs group dynamic https://www.npr.org/sections/thetwo-way/2016/03/21/470870426...

"What happens if our software works exactly how we intend and we gain the ability to really manipulate people in a certain way?"

This is still really difficult, and quite ethically fraught.

When I was in high school and college (late 90s & early 2000s), I was one of the techno-utopians who believed that the Internet would usher in a new era of censorship-resistant free speech, free information, and free exchange of ideas with people across the globe, regardless of whatever walk of life you happened to come from. It would give a voice to marginalized people regardless of how niche their views are. By and large, it succeeded in this, and this site (HackerNews) is an example of that. I have no idea who you are or where you come from, and it could be literally half a world away from me, but I can still post this comment, and probably hundreds to thousands of people will read it.

Yet nowhere on our radar screen was the idea that maybe we wished some of those marginalized groups would stay marginalized. That in addition to giving a voice to dissidents and hobbyists and nerds and subcultures, we would also give a voice to racists and militants and pedophiles and drug traffickers. Why would we even consider them? The definition of "marginalized" involves being "out of sight of the mainstream". Yet now, one of the main complaints being levied against Silicon Valley companies is that they aren't doing enough to police hate speech or kick undesirable groups off their platforms.

And then this is hugely ethically fraught, because I'm sure that from the perspective of people in those groups, I'm the evil powermonger, and they are just exercising their human right to believe what they want. Silicon Valley has remained largely libertarian in outlook, but what if people get their wish and it starts cracking down on undesirables? Who's to say that your group would actually be one of the desirables? The new boss starts to look an awful lot like the old boss.

The idea that how we organize ourselves socially online might lead to echo chambers and deepening division was around long before the phrase "social media" had ever been used.

That this phenomenon came about through the heretofore unprecedented (sarcasm) exploitation of human credulity and vanity, that dopamine spikes are the very mechanism of addiction, and that we react more strongly to things that trip our amygdalae than our cortices, should surprise precisely no-one — least of all the people who were building tools predicated upon those things.

None of this stuff is new. None of this stuff is news. You don't get to play ignorant of the implications of the things you're building. Not while you're selling your time on the basis of how smart you are, anyway.

Perhaps not predictable at the outset, but surely FB developers had insights into the usage/sentiment data long before reporters got onto it?

> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society. It's not a huge leap to go from "I'm pretty sure that certain people will become obsessed with this," to "It's probably harmful for lots of people to become obsessed with this."

I don't agree. I think this causal relationship makes sense in hindsight, but I don't think most people capable of designing or implementing software can reliably extrapolate what its impact on society will be. That goes for both positive and negative impacts.

Researchers and engineers should still put the effort into questioning the nature of their work and its impact on society, because that's a good exercise in general. But I'm skeptical any given individual would be good at arriving to the right answer.

More-over, I don't think parent is giving enough credit to certain types of computer scientists and certain companies.

Facebook, Google, and some game companies definitely do hire lots of human-computer interaction Ph.D.s who definitely are trained in exactly this sort of thing.

There's a reason the title is "computer science researches" and not "computer programmers".

> On the other hand, the people who design Facebook or mobile games to be maximally addicting definitely understand something about how their software is affecting society.

This is an interesting point, a whole lot of companies do actually hire soft-sciences talent specifically for that purpose, has been quite common in the gaming sector for years now (Blizzard, Valve, Bungie, just from the top of my head).

This point backs up the statement of ~this parent~ rossdavidh's reply: programmers and engineers are woefully unequipped to perform any significant social science research. If they were, then tech companies would have little to no need to hire research psychologists or human factors specialists. They'd instead hire programmers and engineers that specialize in getting people addicted to technology.

So because the employer hires specialists in weaponizing people's brains against them (via manipulating addiction psychology, &c), the programmer is somehow off the hook for implementing things that he now knows to be designed thusly, on the basis that it's the work-product of those specialists?

I'm reminded of the Tom Lehrer lyric:

  Once the rockets are up
  Who cares where they come down
  That's not my department
  Says Wernher von Braun

You're counter-argument is disingenuous, almost to the point of willful ignorance.

It's obvious that a missile can be used to send bombs across the world just as well as it can be used to send satellites into space.

Minecraft seems like a great form of entertainment that can be used as an educational tool. The fact that it might also be destructive to the development of children is non-obvious and takes some expertise outside the domain of software engineering to uncover.

I wasn't talking about Minecraft, necessarily. But consider, for example, Zynga, or any of the makers of pay-to-play games that are designed with a thorough understanding of addiction psychology in mind.

You didn't design the mechanism, but you know what it does. Are your hands clean if you write that code?

> If a person can predict the consequences of design decisions well enough to get paid for it, they'll probably have some idea of potential negative consequences as well.

Being able to hypothesize about potential risks and being able to perform experiments to verify the hypothesis are totally different skills.

You can guess that maybe your game will be unhealthy for teenagers, but constructing an experiment to verify this is a whole different ball game.

I think you're way over-estimating what a realistic ability to predict outcomes will be.

I'm reminded of back when I worked in agency land, and clients would ask us to build something to "go viral" back when going viral was a new concept. They assumed it was as easy as just saying "make it so".

Of course, it's never that easy to know that it will catch on.

How exactly do you make something that is fun, engaging, and rewarding enough to encourage regular use, but not fun, engaging or rewarding enough to abuse? Seriously, if you can answer that question, you're sitting on a gold mine of knowledge.

It's much difficult to differentiate between "many people will enjoy and be entertained by this game" and "people will be harmfully obsessed with this game".

I agree it's hard to pick which games will be which, but it's not hard to predict that the more addicting a game is, the more people will be harmfully obsessed with it.

Reply on news.ycombinator.com