City of Amsterdam’s Algorithm Register

It was discontinued because apparently it was felt it didn't allow for enough of the 'subjective' aspect of it. The problem was also a bit that the software had slowly become less 'guiding' and more 'the norm', where judges and prosecutors felt they had to really justify deviating from the outcomes of the software. I imagine that could have been solved with a more tunable parameter for how much to weigh the 'subjective' part, but that would open another can of worms of course; and it's also a philosophical issue, so I imagine that a lot more went on behind the scenes that the general public will never know.

Anyway, that system would never be put under the umbrella 'algorithms' nowadays. That has shifted to ML/stats systems where the 'morally problematic' part of the algorithm are really the 'neural network weights' (or regression coefficients, depending on how impressed you are by raw processing power over system understanding... but that, again, is a philosophical issue).

To rephrase: my position is that the problem people have with what they call 'algorithms' isn't really the algorithms at all (nobody who talks about this problem, at this abstraction level, cares abour gradient descent vs simulated annealing), but the parameters to the resultant models and how those parameters were estimated; specifically what data was used to make those estimates.

I personally really despise this, in my view, dumbing down of the matter and muddying of the terminology and hence issue, but then again nobody asked me when 'hacker' was rebranded from 'a specific type of technical intellectual curiosity' to 'computer criminal', so...

I think there's no other way around this: ultimately, everything that has legal implication will be potentially subject to dispute. We're not yet in the dystopian environment where disputes can be handled by a model :)

The development of these two registers has been a very deliberate process for both cities, to come up with a register that has technical detail as well as makes the algorithms understandable for the masses. If we can improve in any way, please leave your feedback in the survey, it's most welcome!

Suppose only a poor neighbourhood has laws enforced, while in another there is no enforcement. Would that be discriminatory? I think in effect, absolutely. In terms of intentions, it remains to be seen what the reasons were, it could've been a mistake or a matter of optimising resources. But there are clearly limits to this argument, and it is also clearly discriminatory in practice.

For example, racial profiling is very much a form of discrimination. Suppose two ethnicities live in a city, one (A) has a 0.5% serious crime rate, the other (B) has a 2% serious crime rate, on average, and say equal chance of minor offences. Profiling citizens of group B more often, e.g. stop & frisk, would be a more optimal use of police resources than random selection, to stop serious crimes.

Yet, 98% of group B is innocent of serious crimes and is being subjected to more enforcement, which lead to a higher than proportionate conviction rate of minor offences, simply for belonging to a different ethnic group. It also leads to more run-ins with the police, which disturbs your day, risks escalation, makes you feel like a second-class citizen etc.

That's discrimination, because the 98% innocent individuals from group B are being treated more harshly without cause, purely on the basis of their belonging to a group. Discrimination is when an individual is maltreated on the basis of belonging to a group, rather than as an individual.

Yet also 'more optimal/efficient use of resources'. They're not mutually exclusive.

And that's true not just within policing, by the way. If you know that 80% of ethnic group A have empirically been good hires, and 70% of ethnic group B, in the past, and suppose you had many people to choose from to invite for an interview, and then interview. Suppose you have limited resources and selection & interviewing is time-consuming. Does that allow you to 'optimize use of scarce resources and efficiently reject all applicants from group B'? No. It'd be perhaps efficient, but also very much discriminatory.

Of course if you have two neighbourhoods and one has rampant parking abuse, the other not, then indeed it does make sense to allocate enforcement capacity accordingly. But only to a point. And that point must be evaluated. And whether you've reached that point, will be indicated by the data. That data isn't available, yet it's a priori stated that the parking enforcement cannot be discriminatory. OP noted a few possibilities that could indicate it is discriminatory, but we'd need to see the data to draw any conclusions. I think he's right in saying that.

We're replacing systems that are readily obvious to anyone (uniformed human parking attendants) with algorithmic enforcement.

The government has decided that we don't need to know how it works (source code and models) or what the flaws and externalities are (anonymized data on how it's used). Thus something we're paying for whose function was pretty obvious to anyone becomes a black box.

This lack of transparency will become the norm unless it's opposed, and eventually you or your group will become the disproportionate target of some overzealous algorithm.

1) Where is the data that shows each street is equally/proportionally scanned by parking attendant vehicles? There is potential discriminatory policy there.

2) Where is the data that shows that scan-success rate (i.e., percentage of detected cars, for which a plate number could be made out) is distributed normally, rather than e.g. influenced by parking infrastructure which is correlated with socioeconomic factors? Again, potential discriminatory practice.

Other possibilities exist, too.

These may sound far fetched, and you may be right to say that, but the data would show it. It'd be nice to see the data. It cannot be said a priori that 'algorithm just checks the image, so there's no discrimination possible', if there's no word about ensuring that regardless of various factors (e.g. socioeconomic, racial etc), your non-compliance with parking rules has an equal chance of your plate being scanned and fed into the algorithm.

It's the same issue why racial profiling is controversial. You can say 'this computer/algorithm is perfect at assessing non-compliance with a rule or regulation by individual X regardless of e.g. ethnicity', and you could still have discrimination because ethnic group A is fed into this system by way of racial profiling 10x more than ethnic group B. By the same token, we'd want to see where/how-often the scanning vehicles operate, and how their scans may have lower or higher success rates in various situations.

In short, the OCR might be alright, but the entire process/model/architecture (from scanning to issuing a fine, in which OCR is just one of the middle-steps) may not be. That entire model including scanning (photographing) by a computer is described on the page, so saying its non-discriminatory implies they've assessed more than just the OCR step. And if that is the case, they cannot say it's a priori non-discriminatory quite yet without providing more info/data.