Teams like this attract a certain type of person, and it's not a builder. In order to justify your own existence, you have to invent blockers to place in front of people who are actually trying to build things.
In theory these teams could do some good things, in reality it attracts or creates horrible people with INCREDIBLE efficiency / creates a cycle of endless meetings / recommendations... and it is endless because it is in their best interest to have an endless amount of "work" and inject themselves in a way that costs them nothing, and everyone else a great deal.
I figured these teams were the easiest place to find people who provide nothing at all / get in the way of folks doing things and almost always they don't accomplish their goals anyhow. They wouldn't even know if they did accomplish their goals anyway as these groups tend to center everything on their actions / the goal is them doing things endlessly.
Somehow word got back to them about the joke, they blamed the wrong person for the joke, tried to raise a hubbub with HR (to their credit HR told them to pound sand). And then they got all laid off ...
I'm a believer in some sort of cross-validation and quantification. The ethical impact of a technology should be quantified, and addressed if significantly negative (or promoted/incentivized if positive!) -- you can quantify number of users impacted, give various arguments, soft measures and estimates. Then, you can have other teams validate those estimates. After all said and done, you can again evaluate an intervention (on both ethical terms, and on the productivity and profitability of your company), and see if it was good or not. If a team is repeatedly giving bad advice, I think you can then address that (retraining, lay off, etc.?). Ethics is not impossible to do :)
I believe a certain generalization of this is needed for the entire society[1], to externally address good/bad technologies. A good example I cite all the time is Open source contributions. We still haven't found a good mechanism to pay OSS, despite the immense value it brings to society (openness itself has a value that direct sale couldn't capture). I'm a big believer in distributed evaluation and cross-evaluation: if we had distributed entities to evaluate impact of various ventures (for efficiency, properly categorized and subdivided into science, technology, social costs/benefits, environmental costs/benefits, etc.) we could apply the same logic to make society more adaptive, distributed, and less focused only on the single goal of profit. (I plan to elaborate this further, I think we are going to need to "patch" our current system in some sensible and careful ways to get a really good 3rd millennium!)
[1] Previously sketched this here: https://news.ycombinator.com/item?id=28833230
They attract or breed people who see it as a great / easy job simply to provide oversight but have no investment / zero consequences for what happens. They can give tasks, eat up time, it costs them nothing, but costs everyone else tons of time.
They are also free of any consequences ... how do you know such a group did any good? It's entirely for them to define. "These products were ethical, and we made them that way." Easy to say.
IMO these kinds of questions are up to the folks in charge of the products, beyond just a random group deciding "is this ethical" as they actually also have to understand and know them. If the folks making the product can't handle it, that's the problem.
Or worse, you're cutting corners that should not be have been cut.
Now in terms of this particular abuse prevention case, the problem requires a serious social and technical solution that would be expensive for Twitter to implement, and would harm their revenue to boot...
Ditto for financial audit teams and so-called "IT security": all they do is block, I've never had anyone in either function build something or help me work faster, just additional processes and bureaucracy that slows down real work.
edit: I thought my sarcasm would be apparent, but Poe's law strikes again.
They do fill a role that actually needs to be filled, though.
As we grew into a large company and our security team became an entire organization at the company, the org lost all connection with the product and became a more classic blocker-based team. The org lost empathy with the product and was judged on no product metrics, so naturally the culture in the org began to just be saying "no" to engineers all the time. A few of the earlier hires still try to help, but for the most part they just block.
"But, but, but aren't we all on same side?!"
"Oh you sweet innocent child..."
Once this manager understood the rules, they did quite well playing the game.
"Responsible innovation" is not discrete.
In simplest terms, one can judge if security is succeeding with a security audit. Either access is avaiable to the right parties, or not.
One cannot say with certainty if one is being a responsible innovator.
If your products are great bc they are are built quickly on top of a swiss cheese security policy, the security team is à blocker (or at worst, a delayer) of the company bankruptcy from fraud/fines/asset loss.
If responsible innovation is blocking product, its not certain what (if any consequences) there will be if the product is unblocked. Therein lies the problem
This checks all the boxes on Poe's law:
[x] Parody
[x] Reductio ad absurdum intent
[x] Extreme view
[x] Mistaken as sincereThey consistently thought about the nuances of baking privacy into a product in a way that I didn't have the background or time for. Every time I worked with them, they helped me build a better product by helping me tweak the specs in a way that had minimal impact on our users (unless they were a bad actor) and strongly increased privacy protections. It was like having a specialized PM in the room
My boss once did a similar thing with the quality department. Suddenly we sailed through our DFMEAs. Some people do live to block others, but some are trying to do an important job. Engagement usually pays more than just whinging.
It sounds like they were focused on building product too, then, which is not at all what this is about.
The day to day was reading the comms for a bunch of projects, finding places where you could be valuable, then reaching out to a bunch of project owners and convincing them to talk to you about the project.
I had them reach out once because I was working on a project that dealt with data for school children. We were exploring to see if we could use data help teachers intervene early and prevent bullying. They reached out to make sure we understood the legal implications of dealing with data for minors, and that we weren't just building a product that would encourage the administration to pigeonhole students.
After a brief conversation we got signoff and some great feedback on how to support positive outcomes while avoiding the risk of creating something that felt Orwellian
Sounds like something from Silicon Valley, the TV show.
In 2014, for most divisions, the SDET role was rolled into the function of SDEs, reportedly resulting in drastic attrition of the converted SDETs. The exception was the operating systems division, which was reported to have laid off all of its SDETs at that time.
Turns out many kinds of businesses/product teams (including Microsoft's Windows team, I'd say, from personal user experience) need dedicated QA people because many worthwhile tests can't really be automated well enough with a reasonable effort, or if they can, it's something that often breaks and needs regular maintenance.
I suspect that in the Windows case they simply stopped testing those troublesome cases.
If you asked engineers to do a large amount of manual testing, they would either (a) tell you they were and then secretly automate it, (b) simply not do it, or (c) quit.
The bad security teams just show up and expect to say criticize everything and stop other teams without providing workable alternatives.
The problem with having a team with a vague charter (“responsible innovation”) with unclear scope (e.g. not just security or just privacy) is that it’s really hard for them to feel like they’re building foundations for others to build upon. It’s too easy for them to fall back to roles where they see themselves as the gatekeepers and therefore backseat drivers of other teams. It becomes a back door for people to insert themselves into power structures because nobody is entirely sure where their charter begins and where it ends.
We can talk about whether this particular team is effective. We can talk about unclear scope (though, IMO, the scope here is no less clear than "privacy"). But the complaint that teams that put barriers in front of other teams are always bad is insufficient at best and downright dangerous at worst.
Nobody made that complaint.
Please don't misrepresent my statement, which was specifically, "to justify your own existence, you have to put blockers in front of people who are actually trying to build things."
Good security teams build & they help others build as well. They build secure-by-default platforms, secure common libraries, and guidelines for others to follow with confidence. Working with a good security team means you're confident your product will get to market faster because, at launch time, you know you'll be green-lit since you followed the org's security best practices.
That kind of team doesn't have to invent blockers in order to justify its own existence.
Definitely helps you make your point.
You admit that teams in this space can and do often build tools to make things secure by default or private by default, so why couldn't this team be involved inbuilding "responsible" by default?
Your argument here is basically that you believe security and privacy "blockers" are inherently of value, so a team enforcing them isn't 'inventing' blocks, but other teams are.
"Responsible Innovation" is kind of nebulous and I would expect it to be problematic unless given clear rules of engagement and supplied with a strong leader.
That said, somebody needs to be responsible for dealing with the endless stream of products that are deeply flawed out of the box: builtin racism, builtin stalking/harassment tools, "0-day" doxing of existing users ...
Thing is, any way Twitter can innovate in a way that would bring income is likely unethical. Either it will try to increase engagement by polarizing people - increasing heat; selling user data; showing advertising or additional worse things.
The difference between these things and something like a "Responsible Innovation Team" is that the goals of the latter are often poorly defined and amorphous. What does "Responsible Innovation" really even mean? But contrast that with, for example, security goals, which are primarily "don't get breached". Security folks, privacy folks and accessibility folks should all have a very well-defined job to do. For the other "PR-focused" teams, they usually have to make work up to justify their jobs.
Note that, obviously, it's also possible to have bad security people, for example, who only see their job as to throw up roadblocks and demand that your security checklist is ever-growing. I'd even say these are the majority of "security" people. But the good security people who are able to integrate themselves successfully into your product development processes are worth their weight in gold.
Also IME none of these things are binaries -- a product could be more or less secure, privacy-respecting, and accessible, just as it could be seeking to innovate more or less responsibly. Different aspects of security, privacy, accessibility, and responsible innovation will be important at different times, depending on what's happening in the world.
The jobs are never as well-defined as you'd imagine.
- should we do this?
- who do we hurt by doing this?
- oh god people are hurting why are we still doing this?
However if we're critizing from purely a moral standpoint, how is this any different than claiming that cell phone carriers should be preventing this type of thing over phone calls or texts?
For the record, I don't find that to be a convincing argument either but it's the inconsistency of perspective that irks me.
> [getting] the Facebook dating team’s ... to avoid including a filter that would let users target or exclude potential love interests of a particular race
you see, you have to just ignore those people in the feed, you can't filter them, its better and not racist that way. and who knows, you might become not racist if you see a pretty girl/boy you like, but actually that's probably just racist fetishizing
responsible innovation is doing the same dei doublespeak
It would be nice, but would require an inside person to dare make potentially traceable (by their contents alone) public comments so it's unlikely, to know what the team actually tried to do, did do, and what it achieved. Without actual facts the discussion will just end up a free-for-all.
We of course don't know enough of the specifics, because Facebook works hard to keep it that way. But we do know that Facebook has a body count. If you're looking for a "valuable entry point for discussion", maybe start with Facebook's well-documented harms.
Seeing devs non-ironically complain about internal departments like this one which was set up in order not to let that happen again kind of saddens me a lot. No, productivity and work is not more important than making sure that that work doesn’t enable a genocide campaign in a specific corner of the world.
In the context of ensuring a bank doesn't transfer money to terrorists, this is completely wrong. Banks have a whole list of operations and processes, and failing this is enforced by actual jail time. This is why "know your customer" exists in banking. In the context of terrorism, there is no police enforcement regarding terrorists; often, we are talking military enforcement.
This is also incorrect. Transferring money to a terrorist organization is a crime because countries have declared it illegal. Of course well-funded terrorists have victims and enablement of well-funded terrorism has clear victims.
And yes, the government is using military and diplomatic channels to fight terrorism directly-- by ensuring the resources they have access to are limited.
One is that you think a lot of things just shouldn't be enforced, and that we should allow a lot more harm than we do now. Genocide?
The other is that you think we should have a lot more police to take over the harm-reducing regulatory actions now in place. That we should take the tens, maybe hundreds of thousands of social media moderators now working, but make them government employees and give them guns.
I can't decide which is more cartoonishly evil.
Social media moderators exist to protect the public image of the platform, and enforce community guidelines. They should not be burdened with law enforcement simply because we can't be arsed to do proper police work.
Facebook is not like a screwdriver, a simple, neutral tool that is occasionally used to harm. Facebook is an incredibly complex tool for connecting humans, a tool with all sorts of biases that shape the nature and velocity of social interaction.
People have known for decades that online communication systems enable harm. This was a well-understood fact long before Facebook existed. Facebook is morally responsible for that harm (as are the perps, etc, etc). Something they understand perfectly well because they do a lot to mitigate that harm while crowing about how socially responsible they are being.
You might disagree with most ethicists on this, as well as with Facebook itself. But you'll have an uphill struggle. Even the example you pick, vehicles, doesn't work, because car manufacturers have spent decades working to mitigate opportunities for the tools they create to cause harm. Now that cars are getting smarter, that harm reduction will include preventing drivers from running the cars into people.
The only way to objectively agree about responsibility is to use an actor/agent model, where actors are entirely responsible for their actions, and only actions which directly harm others are considered. Otherwise we're discussing which butterflies are responsible for which hurricanes. I'm happy to be wrong here, but I just don't see an alternative framework that can realistically, objectively, draw actionable boundaries around responsibility for action. This by the way is the model that is used in common law.
Facebook being a complex tool strengthens my point. Should providers of complex tools be responsible for every possible use of them? Is it not possible to provide a complex tool "as is" without warranty? Wouldn't constraining tool makers in this way be fundamentally harmful?
> online communication systems enable harm... Facebook is morally responsible for that harm
Everything can be seen to "enable harm". Facebook being morally responsible is not a statement of fact, it's an opinion. Facebook's actions to mitigate are a) to evade/delay regulatory action, b) to maintain their public image or c) by a small group of activist employees. Only a) and b) align with their fiduciary duty to shareholders.
That's incorrect. I'd suggest you start with Dekker's "Field Guide to 'Human Error", which looks at airplane safety. Airplanes are only as safe as they are because many different people and groups see themselves as responsible. Your supposedly "objective" model, if followed, would drastically increase deaths.
The reason airplanes are safe is because the government is doing its job to regulate the space.
[1] https://en.wikipedia.org/wiki/List_of_accidents_and_incident...
I'm not saying the NTSB isn't important. But it's far from the only reason that we have fewer crashes. Government regulation can be helpful in improving standards, but they set a minimum bar. Aviation's high levels of current safety are a collaboration between many, many people. Starting with individual pilots, engineers, and maintenance techs, going up through collaborative relationships, through companies and civil society organizations, and up through national and international regulatory agencies. All of these people are taking responsibility for safety.
If so, how do you explain the catastrophic failures in construction, food safety, and banking prior to the top-down government oversight of those industries?
They serve two different purposes. The former comes out of a zero-sum, adversarial setting where the goal is to figure out who pays for a past harm. The latter comes from a positive-sum collaborative setting where everybody is trying to improve future outcomes.
If I release a new product tomorrow, I'm responsible for what happens. As are the people who use it. But if somebody dies, then liability may be allocated only to one of us or to both in some proportion.
Again, read Dekker.
"morally responsible" ~= "guilty"
But you are still imho (morally) responsible for the deaths occurring out of the use of your product (this is where we would probably disagree). Even if you were not legally guilty.
I like another example that to me clarified the distinction between these concepts better.
Imagine one morning, you open your front door and find a baby having been placed there somewhen during the night. The child is still alive. You are not guilty in any way for the baby's fate, but now that you have seen it you are responsible for ensuring that it gets help. You would be guilty if you would allow it to freeze to death or similar.
This significantly varies by jurisdiction, and isn't settled at all. I don't think being present makes you responsible either. Unappealing as it may seem, you should indeed be able to pull up a chair, have some popcorn, and watch the baby freeze. People should only bear obligations they explicitly consented to. I don't think anyone has the moral authority to impose such an involuntary obligation on anyone else.
Modelling society as a constrained adversarial relationship between fundamentally opposed and competing groups is more accurate than assuming there is "one team" that knows a fundamental "good" or "right" and that the rest of us just need to "see it". People who perform honour killing or preside over slavery are just as sure of their moral superiority as you are. What we need is a world where we can coexist peacefully, not one where we are all united under one religion of moral correctness.
But they wanted to be valued higher, so they explicitly chose to instead step into the editor's role and became responsible for the content on the platform.
What makes you think this possible? I don't see where Facebook is particularly responsible here. Telephones and radios have been used to coordinate assassinations and genocides. Movies have been used to justify invasions. Why isn't anyone burning the effigies of Alexander Graham Bell, Guglielmo Marconi, and Thomas Edison?
In any case, perfection is rarely possible but often an excellent goal to aim for. For example, consider child car fatalities. We might not immediately get it to zero, but that is no reason to say, "Fuck it, they can always have more kids."
edit: I'm sorry, we do get to discuss how they were probably wrecker nutcases stifling people who actually build things in order to make up for their own inadequacies and inability to do the same. It's only assuming that the ethics team worked on ethics that is out of bounds.
Many people are taking it for granted that Facebook should have no interest in reducing harm. I'm glad somebody pushed back on that.
The problem is trying to get people that think everything you listed is obvious and boring to spend 40 hours a week staying out of the way 99% of the time, but also being politically savvy enough to get a few CxO’s fired each year.
Also, since the office is normally doing nothing (unless the office has already completely failed), the people in it need to do all of that, and continuously justify their department’s existence when their quarterly status updates range from “did nothing” to “cut revenue by 5% but potentially avoided congressional hearings 5 years from now” to the career-killing “tattled to the board and tried to get the CEO fired”.
If you know how to hire zero-drama, product- and profit-focused people that can effectively do that sort of thing, consider a job as an executive recruiter.
You can guarantee no harm by doing nothing but that's not a good enough answer.
- should we name this device we want to put in every house after one of the most common American female names?!
engineers and ceo: I see no issue with that!
Several millions people named Alexa now have everyone from toddlers to their friends yelling their name ordering them to do stupid tasks and “Alex stop” repeatedly.
The name cratered in popularity for good reason.
Yet Amazon still has not renamed their dumb speaker
The "Alexa stop" thing, is it a real or invented harm?
My name happens to match the lead character of Karate Kid and I constantly asked to do the crane pose when I was 7. Doesn't seem to have traumatized me.
32nd most popular is not exactly obscure. Why did they have to give the computer a human name in the first place? Probably because it helps people form some sort of parasocial relationship to the product, which is gross, but probably good for business.
Also probably because it was good for business.
That anecdote is nice but honestly it sounds like you survived a less frequent and more temporary somewhat similar but much milder version so now you’re stating everyone else needs to get over themselves?
Your name is now Kleenexified to mean robotic female servant, no harm!
It doesn’t seem like you googled or looked into how people named Alexa actually feel before pronouncing how they should feel.
This comment chain really shows why these responsibility vetting teams are needed, a lot of corporate workers are not empathetic or considerate beyond their immediate siloed task and assume everyone should react exactly the same as they did to only very tangentially similar experiences.
Nonetheless, defaults are incredibly powerful. [1]
[0] https://www.rd.com/list/how-to-change-alexas-name/ [1] https://www.nngroup.com/articles/the-power-of-defaults/
The number we have today minus those unemployed minus those on responsible innovation teams. (Using the broad definition of elites, which captures pretty much everyone who went to college.)
Steve Jobs: "focus is about saying no"
Mark Zuckerberg: "move fast and breaks things"
Facebook is the world's least-trusted, most-reviled brand. They earned it by building garbage.
No one (aside from engineers) cares about what facebook built. Facebook had scandal after scandal and on top of it, everyone's grandparents signed up so teenagers couldn't post all of the screwed up stuff they were doing anymore. Posting there turned into a mix of corporate scrutiny (especially prospective employers) mixed with the old forwards from grandma emails you used to get in the late 90s/early 2000s.
That's why facebook is dying.
At a glance, our comments appear to differ substantively. At closer inspection, the only disagreement we have is what to include in the 'building garbage' bucket.
In case I misused language, I can rephrase my original point: FB is in a bad place today not because FB was ever slow to build, but because what FB did build was too often bad for users (for that matter: bad for humanity).
They built an amazing product... for a world that had only teenagers on social media. This was the truth for quite a while, and was true of the world they created their product in. They just never really updated their product to account for the new reality. Sure, they changed the algorithm a ton and re-tweaked the UI, but the algorithm and the UI weren't the product. The experience people had on the site was the product.
They're sort of like IBM's mainframe division. Mainframes are kind of cool if you ever get to tinker around with their hardware, and they're impressive engineering marvels (so are facebooks systems). But the world has changed since IBM was on top, and they don't fit what most people need anymore.
If I make a pizza with tasty, fresh ingredients - but then top it off with rotten fish guts and mouse turds – it's a garbage pizza.
And those "features" are all fine in a world where the internet is mostly young twenty-somethings with no power, as the internet was in the early-mid 2000s. Also for a product where you can just put in a fake name, as we could back in the day.
those "features" are all fine in a world where the
internet is mostly young twenty-somethings with no power
I closed my FB account in 2011 specifically because the design of the site abused its users in countless outrageous ways (come to think of it: "Are you sure you want to close your account? [These friends] will miss you").
I really don't see us coming to any agreement. It's been fifteen years and, while FB has pumped out plenty of LoC, FB has never shown an iota of respect for its users. No surprise then that the brand is dirt, and Zuckerberg has to spin fairytales about how its 'metaverse' will stem the decline. Their ability to execute is not the problem; the demographics of their user base is not the problem; the product itself is the problem.
That said, it does sound like a strange function to allocate to a specific team.
EDIT: I don't know about these specific people or whether the blockers they put in place are justified. However, I've definitely worked with developers who obviously prefer to just write code until something works with little regard to the reliability, efficiency, security, or maintainability of their output. I spent almost all of my time cleaning up messes rather than getting to build anything new myself. It's a terrible experience.
Yeah, that's not what this team was responsible for, though. That's what an enterprise architecture team ought to be responsible for, but rarely are.
Most "builders" are, if not constrained, like cancer cells. Unlimited consumption & growth without a counterbalance. In order to justify their own existence, they need to continue to churn out things without concern of what they consume, what use their product has.
Maybe, just maybe, there's value in having different kinds of personalities - unchecked by itself, pretty much every tendency is ruinous.
Also, false dichotomy there saying a "responsible innovation" team is needed to limit the building of products with no use; that's a solved problem. We have markets for that.
It is hurting society. What is to be done? That is the issue at the heart of this headline.
1) It's starts after the product launch post-mortem 2) Team creates a process to improve avoiding future mistakes 3) Process innocently includes ability to hard block any product changes without someone or some group's approval 4) Politics emerges behind the scenes from those with social capital to get around the hard blocks 5) New process to make things "more fair" by adding more hard blocking rules, usually resulting in lots of pointless meetings and paperwork to justify new process and show upper management that things are working as intended 6) More politics as teams start hiring for people who are able to get around the heavy process by being "good at the game" 7) New team is formed around the process because it's now a full time job 8) People good at politics join this new team and everyone else is pulled into pointless meetings and handed paperwork 9) Rinse and repeat
The problem I've always observed is that these systems are put in place with the best intent to offer guidance so that company, employees and consumers can all benefit by building the right best thing. However, the systems are given blocking abilities that after time do more damage than good.
You'd think, if something were not blocking no one would listen, and maybe you're right. I definitely have no idea how teams can avoid this spiral trap, however, I don't think it's a bad thing to sunset process like this after a while. Just like Google is willing to throw away products on a whim, we should be willing to throw away process teams on a whim.
It's very much in the spirit of "the bureaucracy is expanding to meet the needs of the expanding bureaucracy". There's no better way for someone to justify their existence than creating problems but appearing to solve problems for other people. Compliance, security, etc. These things are usually necessary but it does attract a certain kind of person who seems predisposed to empire building through obstacle creation.
You see the same thing with people who become moderators. I saw this years ago on Stack Overflow. You see it on Wikipedia. You see it on reddit. You have to be constantly vigilant that these people don't get out of hand. These people end up treating the process as the goal rather than the means to an end.
I remember when the moderators starting closing SO topics as "not constructive". There were a ton of questions like "should I do X or Y?" and the mods decided one day to start closing these became there was no definitive answer. You can list the advantages and disadvantages of each option and list the factors to consider. That can be incredibly value. Can this descend into a flamewar? Absolutely. But just kill the flamewars and the people who start them. No point throwing out the baby with the bath water.
For another, the ever expanding administrative overhead of colleges is a large contributor to driving tuition increases. See https://stanfordreview.org/read-my-lips-no-new-administrator... for an idea of how bad the problem is getting.
I don't think that was a decision made by the mods, I think that's policy created by SO. I really like that it's so focused, clearly defines what is on and off topic, and strictly enforces that policy. You're right, such discussion can be very valuable, but if these policies didn't exist or weren't enforced, I don't think the site would be nearly as useful. Doesn't mean that the people with closed questions are bad, or the conversation can't be had in chat or a different site.
> Can this descend into a flamewar? Absolutely. But just kill the flamewars and the people who start them.
Meanwhile the strong contributors get tired of dealing with (even more) toxicity and move on, the signal:noise ratio drops, and it turns into just another (ostensibly) programming-focused forum. Not to mention that the more time mods spend dealing with toxic behavior, the less time they have for work that improves the content of the site.
There was little to no direction from the top, at least at that time. Mods were just self-appointed community members who organized in such a way to impose their will on the community. It was pretty classic tyranny of the minority. Many of the more prolific contributors (of which I was one at the time) were deeply frustrated by this. It was a frequent topic on Meta SO.
You saw these waves of subtle policy changes if you tracked edits to your contributions. Suddenly there'd be a bunch of edits all based on some new formatting policy and a lot of it was just pointless busywork.
Some people just fall into this trap of elevating the process to where the process becomes a goal and is treated of equal (or even higher) value to the content it's applied to. Moderation is important and has a place but, left to their own devices, moderators will create work for themselves just based on their idle capacity. You need to avoid this.
And then "toxicity" just gets redefined to an always leftward-moving definition of anything "right-wing". Every time.