Many types of data are prohibited for various types of insurance decisions. This varies greatly, from general rules such as banning price discrimination due to race or sex, to specific topics like the ACA[1]'s prohibition against denying coverage due to "pre-existing conditions".

With modern data analysis methods, it is (sometimes) possible to infer prohibited data from other collections of data. For example, various machine learning techniques used on economic values such as income, home value, and proximity to good schools might reveal something about recidivism rates, but it's probably finding a confounding variable such as redlining[2] or other types of institutional bias[3].

This is a problem in both legal decisions and insurance, and transparency in how decisions are made is vital in both. That said, the impact on insurance is probably[4] a relatively minor change in price, which is very different from a Boolean decision about someone's freedom. Even in the unlikely case where the statistical biases are known and accounted for, it still isn't appropriate to over-interpret results about populations as if they apply to any particular individual.

[1] Affordable Care Act ("Obamacare")

[2] https://en.wikipedia.org/wiki/Redlining

[3] https://www.youtube.com/watch?v=qXQA6D4JC0A

[4] for definitions of "probable" somewhere between "an educated guess" and "meh; whatever"

Yes, it is what we do with insurance. The crux of the matter is that driving is a privilege, incarceration is a restriction on/revocation of a person's rights.

Healthcare is a good demonstration of where lines are blurred (at least in the United States). Some view it as a right, others don't.

Not really. It's true that insurance companies charge more to certain people because they are statistically more likely to, for example, be in a car accident than another person or because a home in a particular location is more at risk to wildfires.

But that's where the similarities end.

Insurance companies have a much more and higher quality data than these companies do. More important, the ways the insurance companies price their product is public and subject to competition. If an insurance company realizes that a specific segment of the market is less risky than previously thought they can make a lot of money by dropping the price and underwriting policies for that group.

These courtroom applications use secret algorithms, and frankly, have little incentive to improve them.