I also really like this founders note that we wrote. It’s in our own words, etc: https://integrityinstitute.org/founders-letter
We are recruiting new members! If you work on integrity / trust and safety / antispam / content quality / etc, let’s talk.
How is this effort not a virtue signaling by people that made their fortune on the back of this generation’s cigarettes, and now hoping to get traction on being listened to for fixing the mess they created?
What is not in question is that we should be spending time on tackling these problems.
I think you answered your own question already.
Regarding internal solutions: it’s impossible to overlook the significantly misaligned incentives facing integrity teams. They try to solve problems inherently created by the profit model of the company and what also pays their salary. The parallels to the internal scientist teams at Phillip Morris are uncanny.
As I said, this paradox, and the company kicking the can down the road, has been obvious for years externally and also seemed internally per the Slack (or w/e) leaks post 1/6.
So, for these teams, if they’re staying longer than a quick in and out once realities trumped ones idealism at “going where the problem is” (look, I really do get that impulse), continuing to claim the moral high ground in the way this Institute is so tone deaf. Sorry buddy - I’m on levels.FYI as well, we all know what you got paid to (increasingly ineffectually) support this product. Like there’s been leaks for years of PMs discussing Myanmar genocide inflammation via FB/WA for a few years now. Was that not enough to leave?
This is inescapably how a lot of tech is going to judge this period/company. It went on for too long to claim ignorance otherwise.
Edit: You know what, no it's not at all about an external vs. internal balance, that very much misses the point. It's likely that a meaningful change which preserves what FB (and related) intrinsically is will have to come from an internally-driven fix. The technical abstractions are just too much of a problem for outsiders to grok, then suggest a fix for, and etc. etc. ("Senator, we sell ads...").
It's about understanding where the moral and character capital that leadership, especially transformational leadership, comes from, and understanding how the founders of this have none of it. People that have stayed at FB and profited immensely from the experience, integrity team or not, stayed silent about it, and then start coming out with this publicly now vs. years ago (see: Alex Stamos' example), don't have that capital. And it's so distasteful to see them think that they do via efforts like this. That's the problem.
Coming out publicly over the years has done very little. Even now, with all the attention, it is questionable what will actually change. We are much less interested in virtue signaling, taking the high ground, etc. than working with folks who are interested to carve a path forward. We are dedicating our lives to this work to find solutions over the long-term, whether it is in the spotlight, or not.
I’m not sure how you can read that founders letter, and not see a spotlight grab/virtue signals by people who contributed to and profited from the problem they’re trying now take a leading role in solving. It’s like there is just total refusal to be seen as part of the problem. Blows my mind.
“Coming out publicly has done very little” attitude sort of says all you need to know. Pretty sure the article said also ~”now that Frances came out publicly, we can start this!” Mental gymnastics.
"Frances is exposing a lot of the knobs in the machine that is a modern social media platform," Massachi said. "Now that people know that those knobs exist, we can start having a conversation about what is the science of how these work, what these knobs do and why you would want to turn them in which direction."
It seems public discussion does help, and is even a critical input to this effort. It's just better when others do it first.
Registration is quick and has been done by many a skeleton crew over a Slack DM, AWS/Azure credits abound, free GSuite is available, board charter is boilerplate if you can make a mission statement, free Slack pro tier is often an option, $10k in Google Ads are available.
This is all a few weeks of focused evenings.
Man, words escape me. Tethics indeed. Will stop engaging on this thread.
Solving these problems is pretty trivially simple. Get rid of the news feed, or perhaps make it just friends, reverse chronological order. All of these "integrity teams" only exist because Facebook isn't content just making boatloads of money, it needs to make 10x boatloads of money by getting people to scroll infinitely.
People try to pretend this is some hard, difficult problem, when it's only a problem because Facebook is just as addicted to their users' mindless scrolling as their users are.
Not even just that, they also want us to donate to cover the costs for the good work they’re surely going to be doing
Things like this: https://www.aclu.org/other/aclu-history-taking-stand-free-sp... go a long way.
Today all the major text-based social platforms work in about the same way: we type text into an empty little box that's threaded under another box. There are variations in ranking and flagging, but the basic mechanism is unchanged.
It is striking to me that, in all these years, we have explored only a tiny little corner of the design space. There are no mechanisms to lower the temperature when arguments get intense, for example. Nothing to help keep us from misconstruing comments out of context. Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. And so on. It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely."
Many people have suggested eliminating engagement as a metric and going back to purely chronological feeds. That sounds pretty reasonable given where "engagement" has gotten us so far.
But what if there were such a thing as "healthy engagement"?
The hypothesis is that healthy engagement is achievable. I don't know whether it is, but there's a huge range of design possibilities that we have yet to explore.
Do you know of anyone working on healthy engagement? Is it something you want to work on, or do you have any recommendations on starting an effort in this direction?
Since you asked for links, I think you'd like this talk I gave at Berkman a little while ago: https://cyber.harvard.edu/events/governing-social-media-city
It's about: if we think of social media as a new city, what are the alternatives to hiring tons of cops/censors? What about urban planning, bike lanes, etc?
As for healthy engagement: I think there are a few people in this space. I honestly don't know as many as I'd like. This will be a learning experience for everyone. I think New Public might be doing good work, but I'm not sure!
Hope that helps!
Wikipedia is one, by not "working on" it.
>It's almost as though we sold millions of cars without brakes, everyone is crashing into things and hurting each other, and our response as a society is to throw up our hands and say "Welp, guess humans are just too stupid to drive cars safely."
No, they have brakes, it's just that half of the car owners are punching people who work at Jiffy Lube and screaming that brakers should be killed.
>But what if there were such a thing as "healthy engagement"?
"Engagement" can not ever be healthy because "working on" it relies on manipulation. Curiosity implies an absence of manipulation and a maximum of arbitrary connections.
You want healthy engagement? Create something well-loved that doesn't depend on or require any additional action from the appreciator for them to experience its complete impact.
Oh shit, that's hard! "Well then how about I scare them into clicking on a link that pays me money when they do so, then use an image and/or words to call them inadequate in some aspect of their lives so that they send me more money?" Tomato-tomahto?
This is a second, third, or fourth-tier concern after stuff like white supremacists using platforms for organizing/harassment, governments using them to facilitate genocide, scammers using them to profit off of the pandemic, etc., etc.
"When arguments get intense" completely ignores the incredibly low-hanging fruit: banning networks of bad actors. (They have the data to do this, they just don't want to.)
> Do you know of anyone working on healthy engagement?
Twitter is trying to warn users about "intense" conversations. Unsurprisingly, they suck at it:
https://twitter.com/angryblacklady/status/144754672404668416... https://twitter.com/chadloder/status/1446879163307028481
Discourse, too. It gives a statistic that X% of the posts are yours, and asks you to consider letting other's voice their opinion. It says that after you typed your post. I'm not going to not post, then, but I admit that yes it does remind me to consider spending less time on the platform in general (!).
HN has it, too. Deep threads hide the reply button if posts are made in quick succession.
The orig. comment claimed:
> [...] There are no mechanisms to lower the temperature when arguments get intense, for example.
Addressed above.
> Nothing to help keep us from misconstruing comments out of context.
I don't agree with this. Factchecking occurs, thumb-up/heart on factual content helps. Also, quoting, logic, and linking sources allows to dispute (such) fallacies. These tools are available.
> Nothing to assist us in feeling compassion for the people we're talking to, or understanding their intent as they mean it to be understood. [...]
Bingo.
Because polarized people don't want to. They want to 'win' a discussion instead of learn from it.
Does it rely on some specific research? I mean I'm sure there's a lot of viral stories that are false. But there's also a lot of viral stories that are true - or at least no less true than what you'd commonly see on network TV or read in major newspapers, like NYT. I am not sure why it'd be obvious that something that is shared a lot is necessarily false.
You could imagine us swapping notes, and doing some research as a group, to see if this is indeed an idea that would work. And then spreading that information to everyone, including the platforms themselves.
One way to see what we're doing is glorified spamfighting. Except the spam sometimes isn't fake ray-bans, but is instead doctored videos that call for lynchings in India.
Here's more on the subject. (My ideas, not the full institute): https://cyber.harvard.edu/events/governing-social-media-city
On the one hand, in favor of E2EE, companies can and will use the content of messages, if they have access to them, to micro-target suggested content to users, and this can lead to increased levels of misinformation being promoted to people who have engaged with misinformation. And of course there's the government surveillance angle, which is an entirely separate story!
But if you remove the signals in that content by encrypting in a way that is opaque to the platform, do you substantially reduce the ability to microtarget? Very possibly not, given the amount of graph data the social media company has anyways about group members independent from the content itself. And encryption gives the social media company the ability to wash its hands of any responsibility or awareness of content.
Assuming it were easy to technically achieve (which is a huge leap, to be fair!) do you think it better serves the definition of integrity you've adopted, that a social media platform have the majority of its content end-to-end encrypted, or not?
I don't think we have gotten an institute stance on very much relating to E2EE. Our community advisory board (composed of integrity workers) is the moral core of the organization. So far, when we say "the institute has a stance on X", that has meant "the advisory board signs off on X, and that X represents a good faith consensus view of workers in the industry".
We don't yet has a doc that lays out why we think integrity and privacy can coexist nicely. Speaking only for myself, I think the answer lies in careful design. As an example, you could see, via research and experimentation on FB Messenger, that messages that are forwarded in chains of > N are just empirically overwhelmingly likely to be bad faith, spammy, etc. You could then take that finding to WhatsApp, Signal, etc, and then bake in changes to the UX that make it slightly more annoying to forward messages if they've been on a reshare chain of N/M. That kind of stuff.
There's also some consideration to group size -- if a group is 5000 people large on, say, Telegram, it might be encrypted, but it's no longer really private. Maybe it should be treated differently? Unclear, let's think and research about it.
I think a rough consensus we might move towards is treating messaging differently than broadcast, and also treating broadcast features inside of messaging apps differently than straight up messaging themselves.
But again, those are just some of my more idle thoughts. There are members and fellows who are better experts on this particular subject than I am.
Does that make sense? Is that helpful?
To the point about research and experimentation, it wouldn't be surprising to me if certain high-level people at various platforms are having confidential discussions about "are there technical means to prevent message-level research and experimentation from being possible in the first place."
And it's vital not only that the public/media recognize when there are conflicts of interest at play, but also that well-meaning employees at various platforms have the ability to see "here are the bright lines that a consensus of your peers across the industry believe shouldn't be crossed, and here are constructive talking points that you can use for internal advocacy if you are in a position to 'nudge the path' towards a sustainable way for integrity, privacy, and business/legal priorities to coexist."
It's really, really heartening to see people working through these tough questions and working towards a brighter future. You're doing incredibly necessary work.
Looking forward to see what comes out of this and wishing you and the team all the best luck, thank you.
I also have a personal code of ethics, and hold myself to a very high standard of Personal Integrity.
In my experience, talking about that in the tech community does not end well.
Ethics and Integrity do not seem to be popular topics for discussion in SV.
1. They have no profit and loss, so are going to get sidelined by the higher impact (in any short or medium term) concerns of every department that does.
2. They have no direct control over serious corporate activities, so are going to be considered a distraction at best, interference at worst, by others under their own pressures. If they are noticed at all.
It isn't an accident when universal values are silo'd into departments from the rest of the company. It means leadership doesn't want the rest of the company to waste time or focus on them. But feel like the values need token recognition.
--
It gets even worse if these were governmental departments.
Department of Innovation: enforces acceptable methods and subjects of innovation, i.e. anti-innovation.
Department of Ethics: Reinforces the governments views by justifying them as ethical, and demands lip service to those fictional high ethics.
Department of Trust: You must trust us.
Worst one of all: Department of Truth. Here it is. Don't look elsewhere.
(2) Minor - your "Join Us" link from the Founders Letter page is 404 -- https://integrityinstitute.org/join-us
(1) - I'm not sure. I don't think I'm an expert here. But if you've got 15 years of anti-spam experience, we'd love to have you join us as a member :-)
(2) - Thank you. Fixed!
But also, our values: https://integrityinstitute.org/our-values
Also, succinctly -- this is a real, grassroots thing. We gathered a bunch of friends and coworkers for this big idea of "what if we had best practices and a professional association for integrity work, just like we do for cybersecurity" and then worked for 10 months to do it.
We've tried hard to put all kinds of pieces into place. We're making it a place that is both independent of companies and also safe for current employees of those companies to join.
It's also cool that we can draw on this community to give expert advice to stakeholders (policymakers, journalists, companies, academics, etc). Mad that congress doesn't understand how Instagram works, or whatever? We can explain things -- as people whose training was in looking at the total information ecosystem of a platform.
A goal is to have integrity work be at least as prestigious, valued, and essential as cybersecurity, or software engineering is. A thing where quality matters, and if you do shoddy work you will be called out on it.
Does that help?
For example, in the past 10 to 20 years or so we've seen both the ADL and ACLU morph into institutions that would be unrecognizable to ADL or ACLU staffers from 10 to 20 years ago. No one ever imagined that they would abandoned their classically liberal values and replace those values with the illiberal "liberal" values they practice today.
Assuming that wasn’t the point from the very beginning.
To me it's probably something like this: we can think about an information ecosystem or social platform as a system. "Normal" hacking of the system happens through finding loopholes code. (That's cybersecurity). "Integrity-related" hacking of the system happens through finding loopholes in design and rules.
For some easy examples, that covers things like realizing you can post to 1000 groups in an hour. Or using sockpuppets to give artificial boosts to posts. The attackers aren't hacking code, but are hacking a system of rules, norms, and defaults on that system. (And often, finding the holes between where one part of the system was soldered onto the other).
That's the technical part. Integrity also has a sort of ethical component. I think that's meaningful too.
I also do very strongly think there is a moral/ethical dimension to the work. That's one reason I shy away from "Trust and Safety" label. Integrity work may have started with connotations of "structural integrity", but it also must be done carefully and ethically. It's not just engineering, it's also about doing the right thing. Hence the Hippocratic oath we all take. [1]
What you're basically describing as "integrity hacking" is already around as a discipline "social-cybersecurity." If not the same thing, it's likely a very close peer discipline.
Also, coopting a word like 'integrity' vs. 'platform integrity' or 'cyberspace community integrity' or w/e is a tough call. By doing that, you end up with PR-destruction sentences like "integrity also has a sort of ethical component," which at face value is quite a read w/o the context of how "integrity" is being re-defined here.
https://sites.google.com/view/social-cybersec// https://link.springer.com/article/10.1007/s10588-020-09322-9 https://socialcybersecurity.org/
Personally, I do think the field has a lot to learn from cybersecurity. From my POV, integrity work has three different 90's ancestors: web forum moderation,email/search engine antispam work, and cybersecurity mindsets like risk mitigation rather than risk elimination.
As for the PR stuff -- fair enough! We're all trying to do the best we can with the skills we have. Not every decision will be the right one. Maybe the name was one of those.
I hope you check back in with us in 6-12 months. Once you can judge us more by our work over time, I hope we will have earned your respect.
[1] https://en.wikipedia.org/wiki/Hasbara
[2] https://en.wikipedia.org/wiki/Israel_on_Campus_Coalition#Mis...