It’s always some external thing: “the algorithm”, as if that doesn’t just mean “the way we programmed it”.
Computing systems are do not act on their own. That the system did this was because it was allowed to, or more accurately, told to.
Computing systems are do not act on their own. That the system did this was because it was allowed to, or more accurately, told to.
This is true.
> Computing systems are do not act on their own.
That doesn't mean that computer systems behave the way that we intend them to behave, or even that we really fully understand our own intent!
How much of your own code have you formally verified? (https://en.wikipedia.org/wiki/Formal_verification)
How much of your own code even has a precise enough purpose that the spec of what it's supposed to do is shorter than the length of the code? Such that you could even in theory formally verify that the implementation is in some sense "correct"?
For that matter.... how much of your code has a precise enough purpose that the spec of what it's supposed to do can be written down in formal language at all?
And actually... how much of your code has a precise enough purpose that the spec of what it's supposed to do can be written down in ENGLISH at all?
I don't think something like an "extremism filter" can ever be implemented in a bug-free way, because I don't think there's a precise enough definition of what "bug-free" would even mean.
The problem of people blaming bad outcomes on "the algorithm" is real, and organizations should take responsibility for misclassifications generated by code that they own and operate.
It's unhelpful to pretend like engineers and the organizations they work for have zero agency.
However, it's equally unhelpful to pretend like buggy behavior aligns with the intent of the engineer/organization.
(And in the case of automated moderation, the software getting something wrong really should be considered foreseen consequence anyway. Facebook knew, or should have known, that these systems would have false positives.)
But also, "the computer was told to do the buggy thing" is misleading because it suggests mens rea.
We could certainly say a 'corporation intended' to do something if we found an email from the CEO commanding it, that much is trivial. But what if we think of a corporation as a 'slow AI' or 'China brain'? Might the corporation, viewed that way, have intent that transcends the individual thoughts and desires of the constituent employees? The system may be structured in a way that rewards low level employees for doing things the individual executives would never consider acceptable (for instance, 'lying to the executives'), and which the low level employees don't think is a good idea either. With all the humans personally opposed to some business practice, it might still occur due to the structure of the system. If we personify that system, we could perhaps say the system itself has a will of its own, with objectives alien to any of the humans involved.
A harsh example far worse than Facebook: the Functionalism-Intentionalism debate (https://en.m.wikipedia.org/wiki/Functionalism–intentionalism...) At the end of the day, does it really matter? Whether you subscribe to functionalism or intentionalism in that case, the horrible end result is the same and people need to be held accountable for it.
But we do distinguish in regulated engineering disciplines. A contractor that intentionally uses shoddy materials in a bridge or tunnel is treated differently from a contractor who simply fails to implement legally required QC is treated different from a contractor who implemented legally required QC but laid bad concrete due to operator error that wasn't captured by that legal requirement.
> At the end of the day, does it really matter?
Yes, and the reason it matters is immediate when you try to answer the following questions.
> Whether you subscribe to functionalism or intentionalism in that case, the horrible end result is the same and people need to be held accountable for it.
1. Held accountable how?
2. Toward what end?
--
Facebook has good engineers, but they're not gods. It is not possible to write a perfect -- or perhaps even passably good -- "political extremism" filter.
Shifting blame from "the algorithm" to "the engineers" or "the corporation" completely misses the whole fucking point: there is no spec for "perfectly functional political extremism filter". 0.000% of the people calling for "politically neutral moderation" have any god damn clue how to define the thing that are asking for, even in a natural language, let alone a language precise enough to implement.
Just look at the rhetoric used. People blame in passive voice "corporations" and "engineers", while criticizing others for using passive voice to blame "the algorithm". I'm not saying that we don't need a base level of QC and corporate responsibility. I even think Software people should be folded into Professional Engineering with all of the personal responsibility that entails! But we need to be very realistic about the fact that engineers are not Gods who can Solve Politics.
Hell, even if you erase inherent political tension this is still an impossible task. Can you write down a filter that's perfectly biased toward liberal speech? No.
So, again, Held accountable how? Toward what end? Unless your answer is "purposefully kill all social media including HN", mens rea matters.
In the case of the Nazis, the answers are 1. Executed. 2. Justice. For Facebook, I think executions should be off the table, but the second answer is the same.
Facebook's engineers aren't gods, I get that. Knowing their engineers aren't gods, Facebook proceeded to use them to create imperfect but profitable systems anyway. Systems they knew or should have known would harm society while enriching their shareholders. This is facebook's crime. If they couldn't create moderation systems that operate well at a massive scale, they never should have operated at that scale in the first place. They couldn't, should have known they couldn't, but tried anyway.
The president pro tempore of the US senate was a Segrationist in 2001. Literally, segregationists of the US Senate outlived pets.com
If any component of your blame or solution to the state of western politics includes the words "social media", ... good fucking luck.
You are mistaken. Neither intentionalists nor functionalists excuse the responsibility of anybody, least of all the organization's leadership. To both, the organization and its leaders are to be held responsible for the actions of the organization. The functionalist model does not absolve anybody of guilt.
If you want to see the hazard of demanding proof of intention, look no further than the travesty of justice that occurred in the wake of Enron. Every last Enron executive should have been jailed, but prosecutors had a hell of a time getting any of them convicted of anything because of this misguided obsession with proving intent. And nobody from Arthur Anderson went to prison, for the same reason.
Obsession with intent allows the guilty to trot out the "we didn't mean for this to happen, we're all just idiots." excuse. Without a smoking gun email, that excuse is hard to conclusively disprove. The antidote to that is strict liability; saying that their intentions are irrelevant and punishing them anyway.
Reminder, this is my point: "When software goes wrong, the company that decided to create and operate that software for profit is responsible for whatever negative impact that software might have, regardless of whether these problems were foreseen."
Take medical software as an example: a highly regulated space, that very few SMEs and approximately ZERO open-source projects can afford to enter.
And even in that space, strict liabilities are restricted to the software’s *intended use*. As long as the manufacturer has taken steps to clearly indicate what is appropriate use / misuse of the software, the manufacturer is NOT liable if the software is misused by the user. At that point, the liability shifts to the user of the software instead.
An in-between approach would be GDPR-style regulations that define what is and is not appropriate to do, with proportional penalties for failing to do that: intended vs unintended failure; penalties proportional to company income so it can hurt small and big companies alike without outright killing them on the first few strikes. However there is a cost even to that: such regulations do block valid innovation and they tend to expand and get more complex year over year.
There is no easy way out here that I can see...
Sure. Neither do children, nor pets, nor farm animals, nor automated control systems.
The common thread connecting these examples is: people who are responsible for managing children/animals/PID controllers bear responsibility for the action of these systems, whatever the actions are. Same reasoning should apply to algorithms: don't deploy them unless you're prepared to be held accountable for what they end up doing.
It could easily be a model that takes in features such as "has an admin banned this page" or "has an overwhelming number of users reported this page" and then based on that it identifies problematic keywords that it uses to ban pages.
And then it relies on humans to manually interject when the model fails as it sometimes will.
That were programmed to make decision no one understands and to act on their own.
Correctly moderating Facebook, blocking recalled/bogus/questionable goods on Amazon, or keeping Google's search index pruned of scams and misinformation, are fundamentally problems that requires armies of trained humans, well-thought-out and transparent process, and potentially independent or state oversight to keep it all on the rails.
It's much cheaper for them to buy racks of servers, and occasionally shrug their shoulders and blame algorithms. I'm amazed that legislatures throughout the world have been willing to accept that behaviour for so long. It's not like the problems can be explained away with "it's an insignificant business with no political or commercial influence", or "it's a 6-month-old startup firm that made beginner mistakes because they couldn't afford to do things properly".
Break them all up. Even in a world without Google and Facebook, enterprising nerds will be encoding H.265 via smoke signals to ensure we get our cat videos.
Thinking of Amazon for instance, they pretty much destroyed small local vendors who did much more than just sell goods. They were also access points to local community and culture and raised diversity.
Similar story with Google and Facebook and online news outlets.
It's a question of balance of power. The interplay of society and corporations is not a one way street. Corps offer services to society while they also rely on it for resources and infrastructure and to give it legitimacy and protect its assets.
Among Big Tech firms I believe Microsoft was one of the first to understand and incorporate this and so they self-corrected. Facebook is really late to the party in this regard and I hope in a couple of years we will be able to see the situation with more sober eyes.
Not always. The GDPR has some decent provisions against overuse of machine learning, for example (it is just a shame it is basically unenforceable against global megacorps, and otherwise so shit as a set of law). We should not accept it, and I don;t think this is a fight that is lost. It is fine to use ML, as long as you are prepared to suffer the consequences of the mistakes of your models that you do not check. This is an important principle.
> Even in a world without Google and Facebook, enterprising nerds will be encoding H.265 via smoke signals to ensure we get our cat videos.
Damn right. These are important. I'm concerned this statement isn't entirely serious. While I hope not to have to resort to similar to that, I will if I have to. Cat videos are important damn it.
(Charles Williams' All Hallows' Eve is an extended meditation about just that.)
And in this case, it appears that they aren’t.