New York City Moves to Create Accountability for Algorithms
propublica.org
propublica.org
For example an algorithm is too good at detecting which gov't employee is stealing from the public. Or it tells them they should decrease fines in some area to get better results --but that would affect their budget adversely, etc.
They'll pause the algorithm ? Replace it ? They can't do that ...
The issue is that once you automate something you can't unautomate it without providing the workforce necessary. Because in nearly all cases a bad automated algorithm will far outperform 1% of the required workforce in humans, that's how algorithms win. Not because they beat humans, they don't. They win because they actually take action in 100% of the cases, whatever that number is. A dumb algorithm taking action 100.000 times can easily beat a very hardworking human that takes 100 smart decisions in a lot of cases. So that will be tens to hundreds of people in easy cases, and thousands to tens of thousands in bad cases, and this is New York, easily the size of a decent country. So it'll always be "we need to pause this NOW !", "OK, no problem at all, that'll be $15 million per hour. Under what budget item do we fit this ?" "Erm .... How about you just make it look like it's paused ?"
Keep in mind stopping automation doesn't even just cause damage directly, it will also cause overloading costs onto other departments and even onto private companies. Often in surprising ways. Algorithms respond quickly, under nearly all circumstances, at any time. You wouldn't believe how efficient this makes interacting with organizations. Pausing automation has an enormous and accumulating cost, making the decision impossible.
Also algorithms don't solve corruption (they may make it easier to track though, although there are ways around that).
> takes the decision that an algorithm that is working is unacceptable
"Unacceptable" == "not working".
That's such a cynical, nihilistic way of looking at government. I can only guess people arrive at it after being exposed to a rather superficial look at governments' work over a long time.
In reality, the vast bureaucracy that is government takes thousands of actions every single day, almost all of which are uncontroversial. They work hard to establish procedures minimising uncertainty. The work is far more transparent than any private organisations'. And all decisions are subject to judicial review–with the judiciary having its own, long tradition of thoughtful deliberation and even-handedness.
As one example, the list at https://www.regulations.gov/searchResults?rpp=25&so=DESC&sb=... shows some recent (federal) actions. Note that this list is only the tip of the iceberg, with the most controversial administration in modern history. Yet it is dominated by "Class E Airspace; Revocations: Eaton Rapids, MI" and other items of rather low publicity value.
And the government is so full of abuse, it's just disgusting. The thing is even "unintentionally corrupt" as I call it. Regulations that get important things done (especially where it pertains to hiring, consulting, real estate, ...) by just asking it in the right (and "published") location and person. Then, they tell this to 2-3 companies and the rest have to figure it out on their own. Then, of course, they switch to actually corrupt, and change where they need to ask a few months later.
What specific use cases match your description? What is an example of a "case" in your third paragraph above?
There is no binary choice between "not regulated at all" and "completely regulated".
For example, just implementing a process for public scrutiny of algorithms and datasets may result in those building the algorithms and collecting the data becoming more aware of their biases. (That's "may" with a capital M, though.)
That's true, and despite that it will change things.
This sort of unjustifiable secrecy (the accused absolutely have a right to examine the premises of the accusation) can be regulated. Unfortunately, this law substitutes nebulous criteria which, no matter how worthy, are likely to turn a clear-cut situation into a tar-pit of legal wrangling that the victims cannot afford to enter.
The chief medical examiner is still holding fast on the very dubious claim that these flaws raise no doubts about the convictions in other cases where it was used, another area where I think specific legislation is needed.
> How do you regulate algorithmic learning when we don't fully understand how learning works?
This was not one of those cases. There are, however, cases - and this would be one if it applied - where it is reasonable to say that you can't use it until you can explain how it works.
If it breaks on edge cases, that's important.
But that's the whole problem. If you had something that calculates the the thing algorithm is supposed to do in a better way than the algorithm actually does it, you could just use that as the algorithm.
There is nothing wrong with accountability and am glad they are starting somewhere but until they start holding others like the ones who purposesly destroy on time metrics of transit in order to keep thier salary gravy train choo-chooing, I am not holding my breath.
https://mobile.nytimes.com/2017/11/18/nyregion/new-york-subw...
That's a pretty incredible quote from the article above, since it's an average...
In my field we are highly regulated and must provide the regulatory agencies with an audit trail for all data. I would like to see an audit trail for these algorithms that would allow someone to follow the decision tree and review the outcome of the algorithms.