> Crucially, she said, we should place blame on the human individuals behind the creation and use of the algorithm rather than reify the technology as being the main driver.
> “Systems and algorithms are human-made, and do exactly what they’ve been instructed to do,” she said. “They can act as an easy and sort of cowardly way to take the blame off yourself. What’s important to understand is that often algorithms can use historical data without any proper interpretation of the context surrounding that data. With that pattern, you can only expect social problems like institutional racism to increase. The result is a sort of feedback loop that increases social problems that already exist.”
As someone in an adjacent institution this affair has been frustrating to watch. In my opinion black-box algorithms have no place in government, and for this reason it's critical for the government to develop the expertise to develop and understand models themselves, so they aren't reliant on the trade secrets of contractors.
Interestingly though the administration resigned over this affair, it does not look like they'll suffer much over it in the upcoming election this wednesday. The responsible prime minister is likely to get a fifth term if things go as the polls are predicting.