For example, people from X race might be more likely to commit crimes in the absence of any other information (marginal probability). However, person from X race is no more likely to commit crimes than a person from Y race, conditioned on something else like where they went to school, what they do for a living, etc..
It's important to remember that AI doesn't have context, and just because it's using "data" to make decisions doesn't mean the decisions are unbiased - the underlying data may be biased.
You're not allowed to discriminate on race in housing. But is an AI that determines your creditworthiness for mortgages allowed to discriminate on what you eat, where you go to church, what your favorite music is, etc.? Maybe it doesn't have that data, but it will have one level higher - what store credit cards you have and how much you use them and where you opened them.
If you train an AI on a segregated city with a history of actively discriminatory citizens, where the few people of race X who moved into a not-race-X neighborhood got harassed out and sold before they paid off their mortgages, how easily will the AI conclude that people born in certain neighborhoods are more likely to pay off their mortgage if they avoid certain other neighborhoods?
Is that illegal? (My guess is there's no way to prove to a court, to the court's usual standards, that the AI happened to learn the city's racial tensions.) Should it be illegal, if outright racial discrimination is illegal?
This seems to be the case in London, too, so it's a bigger issue than just the US.
But for things like drug possession, yes, blacks probably get stopped much more and not let off with warnings much less.
It is like with plagues. If you put many people with plague in one area do expect that others will catch it. If the rule to segregate is silly enough do expect a correlation of plague with certain characteristics of people put together and the locations. Or their sizes. Maybe it is "cities" or "presence of slums" not skin colour, combined with overrepresentation of people with certain skin colour in them.
It takes some write genius analysis and experimentation to untangle such complex effects from causes.
Call again when you have an AI that can deal with this. Essentially a researcher AI.
If lack of money causes homicides, we should expect to see all poor rural areas have high homicide rates, too. Maybe that's true.
Another factor seems to be sex. Males commit so much more violence, so perhaps that's the only thing to focus on.
Much as there is much hype about AI and machine learning technologies, all of such systems will be, for the foreseeable future, very simplistic models of what we think we know of intelligence.
As humans, we have blindly forgotten that we know very little about the world around us, including ourselves. We think that we have a handle on reality, but we are just plain ignorant. All the models that we have for deep learning and AI or GAI are barely scratching the surface of what "intelligence" is and means.
We may get some useful tools as a result of the research being undertaken today and what we have undertaken over the last lot of decades. But we have millenia to go before we even scratch the surface of what we understand of the universe around us, let alone understand what intelligence means.
That is reasoning. I’ll agree with your later point that we don’t really know what intelligence is yet, but that’s because reasoning is clearly only one type of intelligence. Current systems are terrible at language, for example (a year ago I would have said “and spatial awareness”, but this is progressing fast and I am no longer sure).
> They cannot recognise in any way that the rules by which they operate or the data they are supplied with are or are not reasonable.
Quite a lot of humans fit that description. For example, consider how much angry disagreement the following questions get: “is climate change real?”, “Brexit, yes or no?”, and “does god exist?” Also consider how many people (angrily!) refuse to believe these questions result in any angry arguments.