This is like machine introduced bias/racisim/castism... we need a new term for that.. and its based on statistically induced pseudo-sciences many times similar to astrology. This is the kind of AI everyone should be afraid of.
1. Create conditions which disadvantage and impoverish a segment of society.
2. Refine those conditions for centuries, continually criminalizing the margins that segment of society is forced to live in.
3. Identify that many of the people in that segment of society are likely to be identified as criminals.
4. Pretend that you're doing math rather than reinforcing generations of deleterious conditioning, completely ignoring the creation of those conditions that led to the probabilities you're identifying.
And science can't be divorced from ethics. These are human pursuits.
Sure, these humans can come up with wordy justifications for their decisions. But there are plenty of intelligent people who employ their intelligence not to arrive at a conclusion, but to justify the conclusion they already arrived at. Legal professionals aren't merely capable of this, they're explicitly trained to be experts at post-hoc justification.
And legal professionals basically ignore all criticism not coming from their own professional class. They are rarely taught any kind of statistics in law school. Nobody wants to discuss math in debate club - answers with hard right and wrong answers are no fun to them.
Your pessimism wrt. modeling may be justified, but you're not nearly pessimistic enough about people or the legal system.
Its not okay to assign risk-score because humans tend to show bias given a quantifiable number. And thats just plain bad.