Former Google Design Ethicist: Big Tech in Schools Is Race to the Bottom (2018) | Hacker News Reader

Former Google Design Ethicist: Big Tech in Schools Is Race to the Bottom (2018)

edsurge.com

113 pointshumanetech45 comments

Software has existed for grading and primary classrooms since the early 2000s. The software was awful, and expensive. When you look at Microsoft's new Office 365 offerings for schools...it makes me pretty enthusiastic for the future. I don't see this as a race to the bottom at all. Though I can see some problems with ecosystem lock-ins. But, US schools have long been teaching for the past, Microsoft and Google's new offerings should help schools to lift themselves out of that rut. There are trade offs, but I think the benefits outweigh the bad. My first computer courses in school were on an Apple, that doesn't mean I used Apple for the rest of my life.

The ethicist is pretty concerned about the "arms race for attention", which seems to go far beyond software that simply grades assignments. Sure, the quality and price of those apps may have been disrupted, but there also seems to be a pseudo-social component to education apps that didn't exist in the early 2000's (at least my school).

I think we can agree that building quality CRUD apps has become much more straightforward in the last several years (which could explain the improved quality of grading apps), but there are many ethical questions around data collection, user-engagement, etc. that extend far beyond basic tooling that existed 20 years ago.

This feels like an extension of the overarching debate around social media in general, which Tristan Harris also criticizes regularly.

Interesting comparison to the food industry's race to the bottom, only to be supplanted at the top by retails such as Trader Joe's. Could Facebook's addictive algorithms eventually be supplanted by something more wholesome? The difference between the food industry and social media is that a single person can switch grocery stores on a whim, but social media platform migration requires a crowd.

Facebook can easily be a dumpster fire, but I've found ways to get a lot of value out of it.

First off, by not letting myself get dragged into bad behavior when provoked, which is hard, when it comes to talking about controversial topics, but primarily by focusing on single topic groups that keep a tight rein on topics of discussion... in other words, rules like "No politics or religion, etc." or "We are here because we like "Star Trek: The Motion Picture" and not to hate on J.J. Abrams", etc.

Even fandom-based groups can be ugly, but by looking for groups where the focus is on the positive rather than the negative, great discussion and even debate can be had without having things degenerate to YouTube comments level. Some of these groups will even have an "entrance exam" asking you a trivia question or for your opinion on the topic so that you must demonstrate you actually know something about what the group discusses, and aren't just there to throw grenades.

I think this is the way people can make good use of the fact that "everyone" is on Facebook. It allows you to create your own little clubhouse, and as with real clubs, some are good and some are bad, but you can look for the good ones and leave the bad ones behind.

tl;dr On Facebook, private groups can be very beneficial.

For me Facebook messenger is the only part that matters. If I don’t know you enough to speak with you on there I will never see your posts, since I very rarely go on the main site. So political opinions from my grandma don’t really enter into my usage

I can see how forcing kids onto FB is a major issue, but the article seemed to imply the same was true for other Big Tech EDU platforms (AFAIK the main ones being Google Classroom and Microsoft Teams for Education)

Do the same issues exist in these platforms?

If a kid starts using a chromebook in elementary school (sometimes as young as kindergarten), they are more inclined to use google’s products later. They don’t even get the opportunity to make a conscious choice in their default tech platform, and google might not be the best choice for them. A kindergartener shouldn’t have to think about the privacy implications of their tech choices in my opinion, so maybe the default choice should be a platform with a better track record.

>maybe the default choice should be a platform with a better track record.

Although I agree it should be, how does one engineer things such that an ethical but less profitable software provider has the leverage to beat less ethical but more profitable providers at selling their products/services?

In many cases, the alternative is no tech at all (or very limited tech), especially for schools located in lower socio-economic areas. This notion of tech companies essentially giving away their products to schools in order to influence future adopters has been going on for 30 years or more. With that said, educators should be made aware of this reality and should be upfront with their students that tech alternatives exist.

how about subsidies and open development for these sorts of software?

seems to be doing ok in the raspberry pi arena, etc...

I think the challenge is that it’s hard to compete with the large vendors on a full stack, turn key solution. It’s one thing for a STEAM focused specialist to pull together a disparate set of open source apps into a workable model but your typical teacher doesn’t have the time to do that and still focus on their day job. Most teachers need technical solutions that are integrated, work 99% of the time, and have a very simple learning curve (not denigrating teachers’ technical abilities, this is more of a time thing - they need to be up and running quickly)

I totally agree! My partner is a 7th grade science teacher. She is more competent from a technical perspective than most of the other teachers at her school and she utilizes Chromebooks, and Google Classroom to its fullest. She is really pleased with them as tools, because the alternatives are of such poor quality. Maybe I should go in and give a mini lesson on the privacy implications of using Google/monopolies in tech.

7th grade is completely reasonable to start opening their eyes to how the real world works - i.e. pretty much every person/company/entity you encounter is trying to manipulate you according to their own agenda. I wonder how we can balance the sense of wonder and optimism of youth with the hard lessons they need to know as they sally forth in the world.

A very good point you make - it's hard to walk the line with regards to keeping it real but not crushing students. For example, teaching climate change can be challenging.

I used mac's as a kid in school in the 80's. I still don't use a mac.

So, if your child goes to a school which has "gone Google", personal information about your student is sent to Google without your consent, and their sole path to success depends on interacting with and sending more data to Google. Students are often required to purchase[1], carry, and use Google-controlled hardware daily as part of their routine.

The student has no choice (and no chance at privacy), because the decision was made for them by a school administrator who was excited about getting $100 laptops in bulk.

It's hard to imagine a more exciting situation to be in if you're Google or Microsoft: You have guaranteed customers who literally are required to buy their products. They're forced to buy in, year after year. And the further the buy-in, the further a school system has invested into Google or Microsoft's education platform, the less ability they have to pivot, since the devices they bought are locked to those platforms, the software they're using is running on those platforms' cloud servers, their curriculum is tied to those platforms, etc.

I think the pox here we need to deal with is vertical integration. We need to put an end to the concept of a single company selling you hardware, providing the software, and locking you in on services.

A fun similar racket is body cams: Taser will give a police department free body cams for every officer, so, you know, nobody has an excuse not to have body cams, as far as the public sees. But those body cams only work with their cloud service (which isn't free), and thanks to the handy fact that evidence has to be retained for many years, Taser can effectively make it impossible to stop subscribing once a department signs up.

[1]The school may or may not purchase it for your student, but you paid for it in your taxes anyways.

>[1]The school may or may not purchase it for your student, but you paid for it in your taxes anyways.

Unless your community has explicitly passed bonds earmarked for school district tech purchases, you’re not paying extra taxes for this. The district is making the decision to use existing funds for tech purchases rather than spending that money on something else. If you don’t agree with this allocation of funds, you are more than welcome to attend a school board meeting to voice your opinion.

There are absolutely accommodations that can be made for parents who don't want certain aspects of the students data to enter the G Suite system. But you have to participate and engage with your district, and not just let the wheels of bureaucracy turn for new student enrollment.

I'd be very curious what you mean here. I can definitely understand the possibility of being entered in under another name or the like, to try and avoid traditional PII, but students are supposed to complete and submit work through Google Classroom as well. Particularly in light of Google's AI work, things like an essay a student submitted is, at the very least, personal data, but very possibly, personally identifiable information. There is no realistic anonymity from just hiding someone's name.

I have significant doubts the school district is going to be particularly tolerant of "grade all students on Google Classrom, but this student needs to be graded by hand and recorded on paper" requests.

I'm an educator in a 'Google' school board. We use google products for all documentation, and it's given me pause to think that Google's ML algorithms will be combing through allegedly confidential documents. Is it appropriate for a 3rd grader's bathroom accidents to be documented 'in the cloud', subject to Google's massive profile-making engine? Google Apps for Education (or whatever the title) makes assurances against targeted advertising for students, but it's anyone's guess as to what happens with student account data after it's exported to a 'grownup' account at graduation.

It's unclear to me whether confidential communication respecting students using Google products meets our legal obligations, given that I'm in Canada and (as far as I know) Google doesn't have any data centers physically located in Canada...

Their algorithmic crawlers go over everything you upload to your Drive, however their promise is that data does not flow to advertisers or people who do not have access to the document.

You can choose to believe that or not, but that's the rub.

If its unclear, people need to speak up and promote alternative platforms

I got a young friend that is quite bright minded. He is an engineer, graduated from a top school. He works for Microsoft selling software, mostly the Office suit to schools ... Meanwhile I'm a FOSS "enthusiast". There are a lot of money in the school enterprise sector.

We applaud it but when big companies support this with tons of resources it's just them trying to drive labor costs down aka lower salaries for us.

Check out Shoshano Zuboff's new book - survelliance economy. pretty fascinating stuff.

Here that same Google design ethicist explains in detail how technology hijacks your mind [1].

TL;DR:

Hijack 1: If You Control the Menu, You Control the Choices. Ask yourself: What’s not on the menu?, Why am I being given these options and not others? Do I know the menu provider’s goals? Is this menu empowering for my original need, or are the choices actually a distraction?

Hijack 2: Make apps behave like Slot Machines - give a variable reward. If you want to maximize addictiveness, link a user’s action (like pulling a lever) with a variable reward. You pull a lever and immediately receive either an enticing reward (a match, a prize!) or nothing. Addictiveness is maximized when the rate of reward is most variable.

Hijack 3: Fear of Missing Something Important (FOMSI). If I convince you that I’m a channel for important information, messages, friendships, or potential sexual opportunities — it will be hard for you to turn me off, unsubscribe, or remove your account — because there is a 1% chance you could be missing something important.

Hijack 4: Social Approval. When you get tagged by my friend, you think s/he made a conscious choice to tag you, when actually s/he just responds to Facebook’s suggestion, not making an independent choice. Thus Facebook controls the multiplier for how often millions of people experience their social approval on the line.

Hijack 5: Social Reciprocity (Tit-for-tat). You follow me — it’s rude not to follow you back. When you receive an invitation from someone to connect, you imagine that person making a conscious choice to invite you, when in reality, they likely unconsciously responded to LinkedIn’s list of suggested c ontacts.

Hijack 6: Bottomless bowls, Infinite Feeds, and Autoplay

Hijack 7: Instant Interruption vs. “Respectful” Delivery. Messages that interrupt people immediately are more persuasive at getting people to respond than messages delivered asynchronously.

Hijack 8: Bundling Your Reasons with Their Reasons. When you you want to look up a Facebook event happening tonight (your reason) the Facebook app doesn’t allow you to access it without first landing on the news feed (their reasons), so Facebook converts every reason you have for using it, into their reason which is to maximize the time you spend consuming things. In an ideal world, apps would always give you a direct way to get what you want separately from what they want.

Hijack 9: Inconvenient Choices. Businesses naturally want to make the choices they want you to make easier, and the choices they don’t want you to make harder. NYTimes.com claims to give you “a free choice” to cancel your digital subscription. But instead of just doing it when you hit “Cancel Subscription,” they force you to call a phone number that’s only open at certain times.

Hijack 10: Forecasting Errors, “Foot in the Door” strategies. People don’t intuitively forecast the true time cost of a click when it’s presented to them. Sales people use “foot in the door” techniques by asking for a small innocuous request to begin with (“just one click”), and escalating from there (“why don’t you stay awhile?”). Virtually all engagement websites use this trick.

===

[1] http://www.tristanharris.com/2016/05/how-technology-hijacks-...

Re: Hijack 2

This is huge in mobile apps. Loot creates, "packs", etc. which means every end result doesn't have a fixed cost, but an "Expected" cost. I.e. if you want a certain item, you don't know how many times you have to buy a $3 pack to get it. You know you would pay $15 for it, though. So you start pulling that lever, hoping you'll get it. It's gambling, essentially. Literally a slot machines where you put $3 in, pull a lever, and maybe you get nothing good, something okay, or something great.

Yes, it does seem to be gambling indeed. Legislation should catch up with this kind of tech and clearly label some of its aspects as gambling and therefore not suitable for children and teens.

Thanks for the summary. Very interesting.

Big tech is a race to the bottom in general.

I think such statement is hazardous.

What if you need to build maps for all the world? What if you want to go to Mars? What if you want to build a payment system?

Small actors could not do that.

Many of us were just fine before google maps and paypal. Many of us are just fine right here on Earth. None of us will be fine when Google/Apple/Facebook levels of up-to-the-minute personal information are in the hands of a bad turn of government--think Chinese "social credit" system. Stazi with AI.

We are rolling the dice toward a level of social control unparalleled in human history, and we are doing this for marginal gains to a standard-of-living which is already quite high.

And it only takes one catastrophe to get that bad turn of government--even in a "democracy." It only took a few bombs for us to throw the Japanese into camps when that was clearly unconstitutional.

Why do we need to go to Mars?

Edit: I realize it’s an unpopular question here, but other than it being cool - why do we need a Mars colony?

Quaid told us to.

Why do we need anything?

Off-site backup

That’s not an argument for Mars, it’s an argument for living somewhere other than just Earth. A distant, low-g, airless, wasteland seems like an odd choice for that. If we’re going to have to terraform, at least Venus has an atmosphere and some tectonic activity, and even space stations or asteroids would be more feasible than Mars.

So why Mars?

Because I'm nearly through the second book of Kim Stanley Robinson's Mars Trilogy...

Traditionally governments used to be responsible for such projects.

Exactly. We still need companies, but just to provide the hardware. Let governments develop the software, and keep companies away from our data.

This is how it worked in the early days of the internet, when DARPA and universities developed the internet protocols, and used commercially available hardware to run their software.

"Let governments develop the software, and keep companies away from our data."

Except that government is really just another company, much bigger and often less accountable, plus it can make its own rules.

There is nothing fundamentally different between a government and a powerful company. They both suffer from the same problems, and succumb to the same evils. If this weren't the case, then we wouldn't need unions for government workers. Governments also don't have competition driving them to increase efficiency or improve their service to "customers".

If the government were in charge, the Internet would probably still look BBS's from the early 80s.

There is definitely a role for government to help create and agree on International standards, and to create regulations to protect people, but pretending that handing this stuff over to them would not be a total train wreck in one way or another is naive, in my opinion.

> If the government were in charge, the Internet would probably still look BBS's from the early 80s.

Did you read my second paragraph/line? The government built the internet as it exists now (minus the corporate junk).

Big everything is a race to the bottom. The trick is to find your niche.

Please don't tease us! Some actual information would be nice.

I figured most people could come up with their own examples easily enough.

Take social media. I know so many people in business and media who did perfectly fine before it. Now that it's table stakes, we all have to waste time on it. Great for Twitter and FB, but just a cost for the rest of us.

Or youtube and the porn sites. If they infringe, what are you going to do about it? They take it down, someone else re-uploads--a full-time job to keep it off, and they're purely on the honor system when it comes to royalties. It's not like you can sic your auditors on Google's server room.

Or ed-tech. So many school districts spending millions on VMs and tablets for every child, yet the academic performance is as mediocre as ever.

And it's like people have no concept of just how dangerous information is--like cracking the enigma machine in WWII. Google is probably sitting on enough info to ruin most of our politicians. What if they were inclined to action?

Reply on news.ycombinator.com