Man Allegedly Raped in Jail After AI Wrongly IDs Him as Suspect Despite Alibi | Hacker News Reader

Man Allegedly Raped in Jail After AI Wrongly IDs Him as Suspect Despite Alibi

ibtimes.sg

105 pointsparkaboy91 comments

And if you're confused about how a system could even be deployed if the accuracy rate is so bad it leads to a majority of those identified as criminals being false positives... This can result from just measuring accuracy rates, instead of sensitivity and specificity or related error metrics. Say a classifier has a 99.9% accuracy rate for any given photo. If used to identify criminals, and if only 0.1% of the population are criminals, this will lead to a false arrest rate of 50%.

It is a basic statistical fallacy literally called the prosecutor's fallacy, because it gets misued so much in criminal law, also called the base rate fallacy. 99.9% accuracy sounds high, and is above what state of the art performance is for noisy real-world surveillance data.

But imagine this system is used on a city with a population of 1 million people. With an accuracy rate of 99.9%, the system will correctly identify a criminal as a criminal 99.9% of the time. But the actual proportion of criminals is, say, 0.1% or 1,000 criminals out of 1 million people.

When the system scans all 1 million people, it correctly identifies about 999 of the 1,000 criminals (99.9% of 1,000). But here's the catch: it also mistakenly identifies 0.1% of the 999,000 innocent people as criminals. That's 999 false positives. So, the system ends up flagging nearly 1,998 individuals (999 true criminals + 999 false positives) as criminals.

Indeed. The most commonly encountered example uses medical tests for cancer as the subject and it is one of the reasons why increased testing does not lead automatically to improved outcomes.

It’s kind of insane that’s we have a justice system that’s so insane that it’s normal to receive “extrajudicial punishment” and that there are people who actively enjoy and secretly, and sometimes not-so-secretly, wish that this mechanism remains in place.

(We should also probably not rely on AI to put people in that place.)

Its common knowledge that in jail or prison, "Bubba over there will make you his bitch". And, this is not just tolerated, but endorsed. Its a way for inmates to keep inmates under control.

But how fortunate, a man raping a man isn't actually "rape". Its just "sexual assault".

a man raping a man is rape in most jurisdictions, its only when women rape men does it become sexual assault or "other sexual violence".

> But how fortunate, a man raping a man isn't actually "rape". Its just "sexual assault".

Why exactly are you making this statement in a thread based on an article that explicitly calls that act "rape"? It directly contradicts your argument.

For me the school system was like that.

You were raped in school???

Yes.

If the school enabled this or in some way facilitated it I hope you sued them into bankruptcy.

If I ever made it as a billionaire I'd start a foundation that found the most outrageous bullying cases and tried to sue school districts out of existence. We'd see some culture change if teachers thought they could lose their pension because some other teacher let bullying happen in their class.

If it was over 25 years ago then "good fucking luck", that's just kind of the way the world worked back then before the harvey weinstein's of the world started to be jailed and things like the me too movement brought these issues to light.

The leadership and police were commonly complicit in these actions with the "boys will be boys, therefore no charges" actions. Attempting to bring lawsuits at that time was generally failure prone and put you at high risk of official oppression.

If churches can be sued into bankruptcy long after the fact I suspect that the same holds for schools.

Paul Graham's "Why Nerds Are Unpopular"[0] article makes multiple references to the similarity between prison and school - especially how they both devolve into popularity contests.

[0] https://www.paulgraham.com/nerds.html

I'm fairly sure Paul hasn't been in prison and from the people that I know that have been in prison, especially in the United States it is nothing at all like school, even if there may be some superficial similarities.

For me being the victim of violence was part of the curriculum and I wound up "graduating" from elementary school the same way Ender Wiggin did.

By killing another kid?

Making an attempt.

Never had a talk with the principal, they just told my parents not to send me back. It was a shame because otherwise I liked that school. That kid was making threats against my life and I told the principal before that if he couldn't stop it I would protect myself.

While Michel Foucault sometimes gets a bad rep, he does actually talk about this in length in his book "Discipline and Punish (1975)" [0], in which the prison ends up becoming the Blueprint for other spaces such as the school, the barracks, the hospital and the factory/workplace. While dense, I would still recommend the text.

[0]: https://en.m.wikipedia.org/wiki/Discipline_and_Punish

"A computer can never be held accountable, so has increasingly been used to make management decisions." — IBM slide from 1979.

AI didn't wrongly convict anyone. People leveraging AI did.

Do you have a link to a source?

I've found this: https://twitter.com/MIT_CSAIL/status/1604884273789603842

That one slide can easily be found via google, but I cannot find the full presentation

That’s not quite the correct quote.

Source: https://twitter.com/MIT_CSAIL/status/1604884273789603842

I believe the 1979 original read "...Therefore a computer must never make a management decision".

https://constelisvoss.com/en-gb/pages/a-computer-can-never-b...

I think I actually like the misquote, as it's more powerful, although the original is more prescriptive.

"Computers don't argue"

And AI didn't cause his rape, the prison system did.

Eh, an abusive system inside of an abusive system. It's best to look at it in its entirety to get a view of the systemic abuse.

This is exactly why computers are being used to make management decisions. Diffusion of responsibility is a hell of a drug.

Can we have an honest conversation about all the people enabling X , not just those making management decisions?

If we design an algorithm that will be used to put people in jail by algorithm [1], is that exempt from culpability? What about the engineer that implements it? If they are not responsible, but the code is buggy and the bugs cause real harm, are they now to blame?

Where is the line that we should not cross ?

[1] https://www.cbsnews.com/news/hertz-to-pay-168-million-in-bog...

> The judge then agreed to dismiss the charges against him.

> However, just hours before he was released from jail

How does a person not just walk out of the courtroom after having charges dismissed against them? This case is harrowing.

Also, there are multiple levels of failures here. Why is a person accused of robbery, who just had charges dismissed against them, in jail in the first place, and furthermore, why are they being housed with violent criminals? Why were they arrested purely on the basis that they look like someone, as determined by an automated system, with no further investigation or oversight?

> Why is a person accused of robbery, who just had charges dismissed against them, in jail in the first place, and furthermore, why are they being housed with violent criminals?

Working backwards: robbery is a violent (or threat of violence) crime. That’s why we permit pretrial detention more liberally for those charged with it; to incapacitate the potential for more violence.

As for why they can’t walk out of the court room, I agree it’s ridiculous. He may have had to claim personal effects at the jail and process exit paperwork. But that should be the jail’s responsibility to deliver to him, not the other way.

This link includes the lawsuit's documentation and accuses more than negligent use of facial recognition software - including manipulating/pre-briefing an employee presented a photo line-up.

https://www.houstonpublicmedia.org/articles/technology/2024/...

Sometimes I think that every square inch within jail/prison buildings should be constantly under video recording so that if any rape occurs, there can be no question as to whether it occurred, so that the rapist can be executed with no risk of a false conviction.

That’s probably not actually a sound policy, but, I think it is important to bring the average number of prison rapes per year to basically zero, very quickly, and this is the first thing I think of towards that goal.

... I suppose even with plenty of footage there may be the possibility of erring in determining whether it was consensual?

> should be constantly under video recording

That's what led to this exact instance. "More surveillance" is not the answer to everything.

Especially because of the low quality of the surveillance process - error prone, abuse prone, never respecting the agreed boundaries, and and and. I believe the problems are not with the surveillance per se, but with the people ordering and handling it. But you can't have one without the other so yes I fully agree: more surveillance as a solution is just wrong.

So many things went wrong here, on so many levels. Our systems are falling apart and we're not doing enough about it. We definitely shouldn't be relying on companies like this to fill the gaps.

I don't really want to take away from the point that this is a very bad incident, but, US law enforcement has always been this broken. There is nothing particularly new about this behavior other than the "+ AI" portion.

One of the things that "innocent until proven guilty" needs to mean is that pre-trial confinement is sufficiently comfortable, safe, and accommodating so as not to be confused for punishment, deterrence, or any other outcome of a judicial proceeding.

Putting pre-trial detainees in the same population as convicted prisoners (ie, county jails) is very strange.

I agree and think that's the main point of the story not AI, however questionnable it's usage may be

Horrendous. If more people took seriously the idea they could be incarcerated on clerical errors, perhaps we would see long-overdue and long-needed reforms in our justice system.

Fully agree. Unfortunately, most people won’t even start to process it until it happens to them. Even if not “wrongly”, being jailed for a minor paperwork issue is scarring enough that I am hypervigilant about cops to this day, nearly a decade after my personal experience with the law.

> perhaps we would see long-overdue and long-needed reforms in our justice system

Go to any HN thread about something bad happening and you’ll find people calling to throw folks in jail. We have a strong retributive instinct in America. There is compromise to be had around the mechanics. But without addressing that root cause, we’re going to be an incarcerative society.

So “AI” is the problem and not poor police processes and lack of oversight in crowded jails. Got it.

Both are problems here but politicians in Texas write the laws in ways that try and protect the taxpayer often entirely denying victims the ability to sue negligent organizations. In this case there is a company without immunity that did something bad that put him where he got raped so he might actually have a chance at getting damages.

Texas is all about might-makes-right brutality first.

More than one bad thing can be bad at the same time.

Police are tools of the corporate elite to project their property. Anything to enable their laziness or cowardice is likely to be embraced, even if it's wrong because qualified immunity.

In the future, they won't read you your bill of rights, they'll read you your terms of service

Hey that's a killer opening line for something.

Sounds like something Philip K. Dick would right.

"Please scroll to bottom of Terms and click 'I Agree' to begin 7-day trial citizenship."

https://www.youtube.com/watch?v=IFe9wiDfb0E - 'Welcome to Life: the singularity, ruined by lawyers'

$10 million seems like very little for this man to be asking for under the circumstances.

What’s crazy is he(60) wasn’t even there for long and was already raped, it must be extreme bad luck or this occurs way more than is reported

Likely the latter. It's a failure of the state that has people in custody that these things happen with such regularity that there are jokes about it and that those perpetrating it and those enabling it are not dealt with. Guards all the way up to the warden of the prison have a responsibility towards the inmates.

Happens all the time. A friend's ex-boyfriend was in the local jail overnight and was raped.

Interesting that they make a point specifying "allegedly". I think all rape claims should be treated as allegations, of course. That's how justice works, figuring out exactly what happened. But articles with male victims have gone from not even calling it rape to now expressing it was "alleged". Other articles would just headline that someone was raped...

The fact that all it takes is one company with a wrong ID to arrest someone is one thing.

But that being taken to jail in America can result in violent prison rape (and worse, that it is often met with jokes and snickers) shows a very sad, disgusting part of our society.

The article says he's suing EssilorLuxottica but I also hope he is able to sue the crap out of the government that incarcerated him.

Sadly, this is not a US-specific result. It happens in prison systems, not only in developing countries, but other advanced countries too. That said, it's not as common as portrayed in movies.

Additionally, statistically most male-on-male sex in prisons is consented. For some proportion of the pop that engages in this consensual activity, may categorize it as rape to avoid stigma in their outside world group affiliation.

In the pen the biggest threat is severe beatings.

Do you have any sources/information to back up your assertions?

I did some quick Googling and everything I found just talked about the severity of the problem in the US, though admittedly that could easily just because I am a person in the US searching in English. E.g. https://www.theguardian.com/commentisfree/cifamerica/2012/fe...

No, no sources other than knowing people who know people in the pen. Also from what I've read here and there, so certainly things could be different in different locations.

US prisons are far more violent than in at least most developed world, and for deliberate policy (many politicians and normal citizens think it works as a deterrence, and it's deserved if you're an inmate).

And I'd verify what percentage of that consensual activity began with a rape.

Reminds me of Please Hold (a 2020 short, on HBO Max); here's a trailer: https://www.youtube.com/watch?v=8MTOyyKKCSg

If we just do something about this horrible facial recognition software, only the guilty will be raped in jail awaiting trial.

That's exactly the ethical and competence level I would expect from a company like Luxottica.

this isn't a failure of some private corporation, it's a failure that the infrastructure in place for judicial punishment listens to private corporations as if they were gospel.

I thought AI is experimental at best. No serious job should be exclusively done by AI.

we're truly living in a dystopian hell.

But at least we have Facebook and Instagram and DoorDash to keep us sated and complacent in the meantime /s

I came here to find comments and I had hoped answers as to just how exactly a man walks into a DMV in Texas and gets arrested for a crime that occurred in California?

Why is nobody asking this question??

It seems like that should be a substantial part of the story.

The man was in California when the robbery occurred, therefore could not have committed it. The robbery did not occur in California.

So it would be okay to rape him in jail if he was in fact correctly convicted?

No, that's a terrible take on what's happened here.

The headline is not only was he arrested but also raped.

It is one thing to get arrested and spend some time in jail if you don't face a life-changing assault. It is another thing if you do.

No. Why do you think that? There's nothing in the article to suggest it.

That's the implicit assumption because a regular "man raped in jail" headline has never made it to the top of HN. People are only concerned here because (1) he was innocent and (2) AI was involved, otherwise the usual reaction from the public at large is "good riddance".

If society as a whole is okay with prison rape then it is also occasionally going to happen to innocent people, and we have to be okay with that as well.

Because this is hacker news and not prison news.

It’s a greater injustice that he is innocent, yes.

That doesn't imply the inverse, a guilty person being raped, is justified. In the same way anyone being raped doesn’t justify another being petty burgled; one is just worse than the other.

Wasn't everyone he was with in pretrial detention? Hence none of them were guilty?

That said I think any jail or prison officials are failing just as badly if their inmates are raped regardless of their convictions.

How would the article be relevant to Hacker News if AI wasn’t involved?

How many other news stories about the horrors of people being raped in jail have you read recently?

And then one news story that does come up is "Isn't it terrible that this innocent person was raped in jail?". Not "Isn't it terrible that this person was raped in jail?" with the fact that they're innocent being in the 8th paragraph of the story.

I think there's definitely an inference to be made that the author probably wouldn't have written the story if the victim didn't appear to be obviously innocent. Clearly implying that the rape of guilty people isn't quite as worthwhile commenting on. Because... that doesn't matter as much?

The article doesn't have to explicitly suggest it. The fact that this article was written at all, when hundreds of articles about incidents which were similar in many ways except one, is suggestive enough in itself.

I don't disagree with your point, but this is a false premise.

Reply on news.ycombinator.com