Actually, it's very simple. You yourself hinted at it:
> Put me in a legal case where I (and only I) have full access...
You had to specify "and only I", otherwise your argument would not make sense. Now suppose everybody has access to the same data, and voilà!, your power just vanished.
Through even if we assume so, how would everyone get equal access to the data, algorithms and processing power? If cases are resolved based on who owns most of those resources can we still claim to have a just legal system?
By traditional system, I mean; who is the best lawyer? One who can most effectively manipulate the jury towards their clients case. It sounds jaded, but mechanically I think it's true, no? And what makes one lawyer better at this? Perhaps training, personal talent, etc.
So in the two scenarios, you have:
1. a lawyer wins because s/he's more talented/trained at manipulation. 2. a lawyer wins because s/he's obtained a better ML software suite for processing data.
Is there a meaningful distinction here? Before ML, you always wanted to hire a "good" lawyer. So there were already meaningful differences between good and bad lawyers. Lawyers always tried to manipulate the jury. Lawyers always tried to sell the facts around their case in such a way as to be most effective.
I'm not sure how ML will alter this. Aside from cranking the dial to 11.
The feedback loop between Jury and ML algorithm can continue beyond the duration of the trial.
Without privacy controls, we will commodify the targeted manipulation of all human action and human choicemaking architectures.
Who benefits from an arms race on choice and decision manipulation?
That's already the case. Poor people get a public defender, which usually means taking the offer. Rich people can jam up the courts for years, and can get out of it.
We already live in a moneyball court system.
I disagree completely. People are far easier to convince or manipulate if you know just a few things about them. For someone trained in appealing to people/making arguments, it'd be all about dropping in the right words and making the right suggestions.
Isn't that what we all do with advertising? It's why (some of) it works so well on Facebook. Everyone has access to the same targeting, some people understand better than others how to make their ads work.