Cops are using AI chatbots to write crime reports. Will they hold up in court?
apnews.com
apnews.com
That’s a fundamental issue with so many of the proposed use cases for these things. You can’t throw an AI in jail for filing a false police report. Yes you can simply say officer must review but we all know folks are going to cut corners. Until someone goes to jail for filing a false police report and fails with a “but the AI did it” defense, I’d expect full blown shenanigans here.
We got a preview of this with the lawyer that filed an error ridden court filing and the judge threw the book at the author attorney and wasn’t having any of his “but the AI wrote it” defense.
http://www.nytimes.com/2021/08/28/us/false-police-statements... / https://archive.is/1fCet
Perhaps the AI transcription isn't such a bad idea if the original output is preserved as a diff to the policeman's report.
For example, there may be things the officer (claims) they quietly observed outside the scope of the camera. There needs to be a record of any of those which may be significant. ("Oh, that happened, but blame the LLM for the fact that it's not in the record.") It also extends to "I was justified doing X because Y" logic from the brain of the officer, even if generated after-the-fact.
Just imagine the LLM as Microsoft Clippy: "Hi! It looks like you're trying to retroactively justify an unreasonable search!"
I wouldn't mind a Federal Court -level LLM that flags anything that looks like parallel construction.
I suppose my point is, its never quite that simple and it never seems to work out in favor of the citizen.
Who is going to fund the dispersement of AI to all of these cops, whose budgets btw have drastically increased? Citizens. citation: https://abcnews.go.com/US/defunding-claims-police-funding-in...
Who pays for the lawsuits when they mess up? Mostly, overwhelmingly, citizens. Both for their defense + win and for their defense + loss of the case
What is the real impact of a force that in the early aughts literally shot its own citizens during Katrina? That at the UCs were quelling protests in the most destructive ways possible (also costing citizens). Unclear to me. citation: https://en.wikipedia.org/wiki/Danziger_Bridge_shootings citation: https://www.latimes.com/california/story/2024-07-18/uc-unvei...
I want cops as an idea to work. I have no interest in 'defending my land' or whatever. But the way it is now, plus AI? - no thank you.
Every single sentence was a complete lie, pinning all the blame on me, saying I admitted I was wrong, saying I didn't yield, just complete nonsense. In reality a car changed lanes into me trying to make an exit. My insurance adjuster empathetically told me it happens all the time. They want a clean and tidy story, so they bullshit and hallucinate to throw one party under the bus. No wonder they'd love LLMs.
I can't imagine any scenario where a court wouldn't throw out the entire testimony tho.
If you read more than a couple, just as a casual person that follows a case or two online, it's immediately obvious that they essentially copy/paste or use templates for basically all of them. The reports often state 'facts' that they 'observed' that obviously didn't happen, which are the same things they've happened to observe at every traffic stop or similar interaction. Their testimony in trials is often similar, where they are just repeating canned phrases that may or may not actually be relevant to the specific case. The use of dash cam and body cam footage, as well as trial footage as made it obvious how often this is the case. Their recollections at trial almost never match the footage, and don't match in way where it's obvious that they are fabricating their testimony not just being fuzzy on which details they remember.
So many current fears around AI are based on the fact they might produce an error, as though the status quo is error free. It's not in most fields.
I have no idea if that's how it works now or how well that works, but it seems like the team building this is aware of that possibility and at least taking some steps to try and prevent that from happening.
This is an interesting safety feature but it would be better if the report submission went through and then resulted in disciplinary measures for the officer who submitted it. Simply blocking the submission makes it too easy to keep submitting the report with small edits in between, while never carefully verifying the full report.
The chatbot result should be alongside the video, with links to the appropriate section of the video. Then the defense can look for discrepancies and the judge can easily check them.
i.e. if the AI makes up some shit and someone brings this to a court house, then perjury should apply as usual.
Unfortunately, it basically never applies even in the most blatant cases. Judges and prosecutors aren't trying to jam up their police buddies.
>being held responsible for anything at all
I sure wish.
Otherwise, you may forget important details and just think it was correct if you need to reference it later. We know how influential someone's memory can be.
There is something to be said about needing to write this stuff down fully to commit it to memory.
I agree that this is likely going to end badly.
Honestly I don't think this will be any worse than police filing hand-written reports rife with their own biases and material omissions and sometimes even outright, intentional lies.
If arbitrarily adding more reports is valuable, why stop at just one LLM report? You could include the cop report and a dozen LLM reports with various temperatures and system prompts such as “write this report in the style of Sherlock Holmes as portrayed by Benedict Cumberbatch” or “as portrayed by Johnny Lee Miller”
Edit typo
These LLM’s that can perform actions that look like conversing and thinking always seem to prompt the question: why did we do it that way? IMO the point of an interview is often not to just get the questions answered, but to get a second set of eyeballs with responsibility, agency, and the ability to blow the whistle, to scrutinize the series of events. Unless an AI is created with those abilities, I don’t see the point in having it interview people.
All it would take is some evidence of bias in the follow up questions to invalidate that approach completely.
The article even mentions officers being more thoughtful with words during the stops so that it can be summarized easily later on.
>“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” said Ferguson, a law professor at American University working on what’s expected to be the first law review article on the emerging technology.
>Ferguson said a police report is important in determining whether an officer’s suspicion “justifies someone’s loss of liberty.” It’s sometimes the only testimony a judge sees, especially for misdemeanor crimes.
With the worry being that an officer might not catch an error that the LLM hallucinated or that they'll just overly trust the AI and not give it earnest review.
That said, this sort of summarization seems to be something LLMs do well, I suspect it'll be more accurate than humans in the long run if it isn't already.
1. The majority of criminal cases do not actually go to trial in the US. The charges are bargained down in plea deals.
2. In cases where the case does go to court, if an AI has made a mistake comprehending the recordings, or bullshitted its way through writing a crime report, that puts the onus on the defendant to actually find the mistake.
LLMs are masters at bullshitting, you need a critical eye to spot the mistakes. Furthermore, those mistakes don't just need to be found, they actually need to be fed back into fine-tuning, or they don't get fixed. And there's no way any of that happens in this scenario.
Whatever the cop is feeding to the AI, if the AI can make a narrative out of it, then that thing is what should be in the record.
A loop where the cop generates prose with the help of an AI is bad because
* the AI might lead the cop
* more superfluous prose (which is all the AI can possibly generate because it doesn’t have any actual information about the case other than what the cop feeds it) is bad, it just wastes people’s time.
The point isn’t to create beautiful stories, it is to make a record of the cop’s observations.
It might be the case that we’re making the cops generate a bunch of prose. If that’s the case:
1) maybe it is done for a reason (like to make them sit down and consider what they observed). Circumventing that by just having them jot down off the cuff stuff and having an AI do the “sit down and consider” part defeats the whole process.
2) maybe it is a stupid task and we should consider just asking them for a couple bullet points.
If a bland list of facts is your report, this should be what is admitted in evidence. Reports don't need to be fancy.
I for one welcome new technology to reduce human error and biases.
This I don't understand:
> He said automating those reports will “ease the police’s ability to harass, surveil and inflict violence on community members. While making the cop’s job easier, it makes Black and brown people’s lives harder.”
This premise is anything that makes a police officers job easier means they can do more policing, which equates to making the lives of some people worse?
That assumes the technology does not contain those very biases, which we know is exactly what happens with LLMs. Maybe Axon have fair and unbiased datasets but unless they are forced to reveal that in court, no one will ever know.
Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias.
To what though? Datasets are not easy to create - why do you think they are almost all based on archives of stolen books and videos? Is Axon really going to spend millions to curate billions of words of English conversations free of racial bias for example? The same goes for voice recognition. Is the model for a woman with a thick regional accent ever going to be as good as a man speaking British English without significant investment?
> Just look at how Google got their LLMs to re-imagine the founding fathers to remove bias.
I can't tell if this is sarcasm or not since it is such a perfect example of unintended consequences and hubris.
Every example of bias I can imagine is infinitely more difficult to address with humans than AI models.
If policing has a net negative impact on some people's lives, more policing will be more negative. I can get behind that.
Also, what's with "Black and brown"? Why is only one capitalized?
Of course, the reasoning was just thrown out almost immediately and all previously black people are now Black, regardless of those social or cultural reasons. For example, the AP will label any black person in America as Black, even if they have no shared cultural or social experience in America with people who are descended from slaves(ie a black Nigerian arriving in NYC yesterday is now Black)
I think the problem is that this premise itself is a big pill to swallow. Do people really believe this to be the case? I'm a minority who holds a severe dislike for cops in general, but I don't think this is something I'd ever agree with.
Eg if my house gets burgled, they'll never find anything, but they have hassled me for laughing at something unrelated because they thought I was laughing at them.
Cops will often gaslight victims of crimes in an attempt to not have to do paperwork. I witnessed a man attempt to steal a car while the owner was unloading supplies into his store (a pizza place in NYC). He failed to steal it only because he couldn't get it into gear(a manual transmission), and probably damaged the clutch a bit.
when the cops showed up, it was clear the cops did not want to write it up, and went as far to accuse the owner of lying(and actual phrase said by the cops: "how do we know you didn't say he could sit in your car?", and threatening him with making a false report, or assaulting the thief by holding him until the cops showed up if he pushed it.
Of course, there could be other reasons for not doing it. Maybe they're under pressure from their commander to not take crime reports so the crime rate appears lower in the neighborhood than it actually is. But I assumed it was garden variety professional laziness.
But writing police reports is a nuts and bolts idea liable to introduce way worse problems than what it purports to solve.
Even if this did free up an officer's time, it's not guaranteed that would go to better policing. The department might see it as license to create heavier processes and more paperwork. These reporting requirements don't come out of nowhere, there are strong incentives to create visibility and accountability and they won't disappear if you make paperwork cheaper.
https://slatestarcodex.com/2018/08/30/bureaucracy-as-active-...
Human liabilities for what they do with LLMs is also a question when it comes to defamation/libel/slander.
[0] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4546063
What does "responsible" mean here? How good are we currently at holding police "responsible" for incorrect reports?
This seems like a moral hazard. I believe most police officers want to be good people, and since writing a truthful report by hand is easier than writing a lie (why bother making up a lie?) most reports are truthful.
But, if a LLM makes it easier to just fill in bullshit--and LLMs are very very good at filling in bullshit, if nothing else--then there is a moral hazard. Police will have a personal incentive to allow LLM fluff in their reports.
If the meaningful choices have been made before the AI is engaged -- which seems likely -- then this might have the opposite effect. For example, a few years back one of my coworkers was arrested on a domestic violence complaint. We searched up the charges and were surprised to find an extremely lurid and descriptive account -- right alongside a dozen other identical-but-for-the-name accounts for others arrested on domestic violence complaints that night. Clearly the local police department just had a "domestic violence template" into which they dropped peoples' names without much thought. This level of carelessness would have been much less obvious if they had a LLM change up the details each time.
From one perspective, there should be no harm in simply using LLMs as a text processor, if the accounts written down in said text are genuine and verified as such by the officer.
From a more practical perspective, an account written by a witness itself will always be more accurate to what they actually saw and how they interpreted it, than a statement written by any third party. It doesn’t matter if the third party is human or not.
Also, LLMs show heavy Silicon Valley ethical bias. This is not news to anyone here. To ingest this bias into our legal system, I think grants too much power to the tech companies. Especially in common law, where precedents can be established.
On the other hand, the justice system will be largely supervised by humans, so perhaps they will discern what is right and wrong, moral and immoral, ethical and unethical, or what has perverted ethics.
Then again, if this system becomes more AI-based over time, we may lose human control and our legal system may be in large part controlled by tech companies. This is not a good slope to slide down, and how slippery it is remains to be seen.
Also, what if the halo effect created by the way LLM expresses itself hides institutional prejudice? You can always push ChatGPT to come up with 20 polite ways to express your racist beliefs, for example. But if you wrote your own statements, they may be more visible. I’m sure police already know whatever LLMs produce is more PR friendly as a benefit, but is there a dark side to this?
Lastly, what about data protection? What if I don’t want my data to be ingested by LLMs for training, but I am, let’s say, a victim of a crime? Do I have no reasonable expectation of privacy anymore? Remember what Google said about people who hand over their data to third parties and how they argued it is evidence that such people don’t have a reasonable expectation of privacy.
The responsible adult thing to do would be to run research on it and find how the LLMs bias police statements and what kind of effects that can have to a justice system — proper double blind studies. Done by a committee of experts from judges to institutes and projects for civil justice.
With so many ways to slip off the thin tightrope where everyone acts with integrity and not only best interest, but also competent best interest of one another and justice… I think this is bound to end poorly if we go balls to the walls with this sort of thing.
And then to top this off, other commenters present strong arguments that further complicate the matter.
1) Most police reports are never read after they are written. Literally by nobody. Not the cop, not their superior, not the prosecutor, not the defense, not the court. Most criminal cases end in plea deals and the police reports will never be pulled or looked at. Most charges are filed by an on-duty low-level prosecutor who fields phone calls all day from officers who re-iterate the facts of the crime and ask for the statutes that have been violated.
2) Police have to testify at a trial, usually from memory. This often happens one, two, three years or more after they wrote the report. They are usually handed their old report outside the courtroom before they go in to refresh their memory. This is problematic as they will naturally conform their sworn testimony to whatever is written on the report.
3) Police reports right now are VERY badly written. They are impossibly short, impossibly vague and ridiculously low on detail. If you go back to some code you wrote last year and wonder what the hell it does because you didn't comment any of it, it's a bit like that.
4) Police reports are usually horseshit, for various reasons. They tell one side of a story. The suspect will have another view. The truth generally lies somewhere in the middle.
5) I think this might help, because the AI is naturally more neutral than an officer who's whole job is to apprehend criminals and their natural tendency is to make themselves look good and just and right and the suspect look like a really horrible human being.
6) The problem is that the AI might mishear, misunderstand or just plain hallucinate. At first the officers will re-read it, but after a few of these they will get lazy and just click OK on every report. I've been using AI to generate Alt Text for images and I've been getting lazy at checking it. Just yesterday it told me this cup[0] said "COFFEE" on it, which is a good guess, but it actually says OCEAN FEELS. It guessed, and it was 100% sure it was right.
Just my two centimes.
So running that through an LLM--or even delegating it to a human assistant--is kind of like tampering. It can easily generates falsehoods for a jury, like "this officer has clear recall of events" or "the officer is diligent in fact-checking" or "the officer had a proper reason for what they did".
Sure, an author could try to get the same result on their own, but at least it's the same brain and not ghostwritten by someone who wasn't there, and they can't just shift the blame for contradictions onto The Magic Mystery Machine.