It's great these models are legible and open. Looks like civil service doing a job as well as possible.
The other half of the story is how, in practice these models are interpreted and what actions follow. Are further modification of the model a direct result of that and how iterative is it?
Doing that aggressively changes a bare "model" into an investigative tool, not simply a thresholding utility. Sorry if I missed it, but I don't remember reading anything about how it feeds back. But what I did read was unsettling - entering people's homes at random, quizzing neighbours, pretty fascist stuff. am I wrong?
I guess my point is that if you look at a bare data set, or even an algorithm, sure you can probably infer a lot about biases and intent that might be built in, but you can't see the bigger model within which this functions - and that's the real story.
Is it a persecutory investigative tool?
Like Hicks said we could use cruise missiles to drop food into the mouths of hungry people... a benefit system that could identify people who are struggling (of which crime is an indicator), and, I dunno, give them some money and help? That would be preachy, and quite possible imho.
This is how the system should function, but then you get public outcry when the worst of us are found abusing the system of subsidies while perpetrating their crimes. I think specifically of Marc Dutroux here.
On a side note, after the mass influx of Ukrainian refugees and wide public support, it is much more obvious that those in the greatest need tend to demand less, not more.
I have limited resources for investigating fraud. I need to allocate those resources efficiently. If profile A is more likely to commit fraud than profile B, why would I allocate an equal amount of resources for investigating fraud between them?
I'm going on a hunting trip and have limited time and resources for hunting. I need to plan my trip accordingly. If deer are more abundant for hunting in Colorado, why would I go looking for them in the Sahara?
Am I wrong to look for deer in Colorado rather than the Sahara? Am I biased for investigating fraud where it's more likely to occur? Am I evil for investigating robberies where they're reported?
I have little time and resources on this planet and want to allocate my time and resources as efficiently as I can in order to do the most Good. Why would I allocate my time and resources inefficiently?
By requesting welfare money, you agree that the government can investigate you.
Governments own lawyers advised them that their plans were illegal, in advance.
First-hand experience here. Doesnt work like that.
How does an investigation equal abuse?
It does not in theory but in practice that is the vehicle. You have no idea what Ive been through lately via KYC in France. I was worried Id never see my money while they were making a fool out of me. It's the vehicle and if one's not a discrimination target indeed it's invisible.
I've been there several times.
I needed to provide ample of documents that took time to assemble and I needed to take part on in person hearings, once in a town 200 km away from my residence.
Yes, they pay for the travel and by law my employer needs to provide me extra vacation for those days, but being the guy who needs to take 2 more days again doesn't necessarily boost your career and I'm sure my family would had appreciated if I spend the weekends with them instead of reading up legalisation.
Neither case I've done anything wrong and I was always very cooperative.
I broke, when I was fined for tax evasion, presented evidence on 3 hearings that they were wrong and after they revoked the fine, I still needed to pay interest as according to law I should had paid the fine while the appeal was in progress and since I didn't, I need to pay a surcharge for the delay. Given the entire case was a huge government error I could had appeal against the surcharge, but I didn't want to go through 3 more hearings.
You can absolutely bankrupt entire families both financially and emotionally with this shit.
Don't worry about it when I ask your friends and coworkers about your internet browsing habits.
And oh, after all that I'll not tell anyone "Sorry, we had a mistake in the algorithm and DeathArrow had nothing to do with it at all".
No one thinks this. People that are worried are worried __because__ you don't need AI. AI is just more powerful tools.
- Enables significant increase in surveillance
- Enables highly specific targeted action (i.e. micro-targeted advertising, but not necessarily ads)
- Enables said action to be performed at high speeds with low latency and for cheap.
- Obfuscates decision making processes making it harder to point to who is doing wrong and even how they are doing wrong (because burden of proof is on those being wronged. More black box = more better)
- Enables easier deflection and laziness as one can say it is just math and pretend that math is objective and not subject to usage. You can deflect claims of targeting specific groups because you do not use a variable explicitly defining those groups but you use some or all of the strongly correlating variables.
There's plenty more here too. Of course this isn't all even necessary. I do work in ML and I do have a passion for it. These tools can be great and do a lot to make human lives better. I really do think with them we are capable of doing amazing feats such as reaching a mostly post-scarce society within our lifetimes. You could have them grow the food, pick the food, transport the food, stock the shelves, be the cashiers, and even transport the people (or products) to their desired locations. We could liberate most humans of most labor. That is on the horizon. But even such an amazing outcome which would allow humans to be more human than they've ever had the ability to be in all of history -- allowing them to pursue arts, sciences, community development, and all sorts of things without needing to worry about sustaining one's survival via one's labor. The transition to this post scarce world is not just technological challenges as those jobs won't be homogeneously displaced and as we must adapt to this post world. But I'm sure everyone would absolutely love robot butlers (that are non-sentient. Sentient machines shouldn't be forced laborers, but that's a whole other conversation because sentience isn't required for this. Narrow AI can do many of these already at high competency levels). So even the purely good side is still treacherous.
I'm just saying, we need to think deep and carefully. The nuance is not just digging in the weeds, it is the critical aspect. Ignoring it isn't "good enough" when ignoring the details is what specifically leads to trouble. But it's common to hear that these are claims of pedanticism.