Hacker News Reader

The error is more fundamental. Even if they had some number like 'only 1% of crimes were predicted and that's bad', that's a right answer to a wrong question. Why do they think 1% is not good enough? How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything.

(The right number is probably extremely small, because crime is very bad {{citation needed}} and even a small chance of prevention is useful.)

the "right number" is strongly dependent on what the consequences of false positives are. If you're comfortable pulling numbers from thin air, 0.5% of these guided patrols lead to prevented crimes, but 10% lead to arrests of simply suspicious looking people (suspicious on grounds of being around where crime is predicted) and then while they're being detained are late for a job and get fired. Is 20 people getting fired worth preventing one crime? Say, a catalytic converter being stolen, since "crime" is very bad, not, say, murder particularly.

Further, who in the general vicinity of a precrime report is “suspicious looking”? Who gets their day, if not their life, turned upside down?

Thankfully, police are notoriously fair-minded and empathetic, and never arbitrary, deceptive, or biased when it comes to probable cause.

> How big does it need to be before it is good? 2%? 50%? 100%? If you can't give any answer to that question, then it doesn't matter what the number really is because the number still doesn't mean anything.

Uh, seems a lot like it should be the government answering that question before spending tax money on it and subjecting real people to the uncaring US justice system just because crime is bad so we have to do something, even if that something is ML snake oil?

This is actually the real problem. Police patrols have to be directed somewhere, somehow. It's either the whim of the commisioner, or gut feeling of the officers, or machine learning software.

When one says that machine learning software is terrible, you have to trust either the officers, or their boss. I have a feeling that the same people who criticize the software trust rank-and-file officers even less..

The things you're citing are referring to two different investigations. One is the most recent one that only centered on Plainfield, NJ, which is what Gizmodo is reblogging. The one where they did not investigate Geolitica's effectiveness at predictions was a broader investigation in 2021.

But even if they had used it as intended, how could you tell whether it works? How do you tell the difference between "there were going to be crimes there, but you patrolled there, and because you patrolled, there were no crimes" and "there were not going to be crimes there, whether you patrolled or not, so your patrol did exactly nothing."

Experimentation and tracking results over time.

well, if you're a guest on https://www.probablecausation.com/, then you would randomly assign teams to patrol using this software or not,and compare the changes in rates. sounds like geolitica isn't very careful or interested in results.

If you’re selling tiger-repellent rocks, the last thing you need is a solid controlled experiment to get in the way of separating rubes from their cash.

There are really two questions here, which should be tested separately:

(1) Is the software significantly predictive? Test the software's predictions against actual crime in areas with no patrols.

(2) Are patrols effective deterrents to crime? Observe the area in question with and without patrols. (I wouldn't be surprised to learn there are already such studies.)

If both (1) and (2) yield positive results, you can then use the software to direct patrols and see if the method itself is effective at reducing crime; which would also serve to further confirm (2).

Of course, even if only (1) yields positive results, there would probably be other benefits to using the software to direct patrols, like reducing response times (which could also be tested).

how could you tell whether it works?

The study of those questions and the reliability of your answers to those questions is called statistics and it is an entire branch of mathematics, usually with its own department at most universities.

if crimes (print "there are crimes anyways, the police are ineffective!") else (print "there is no crime, the police are ineffective!")

They used advanced software to predict where clicks will happen with bait.

Maybe the link for this submission should be replaced with one of the originals? The closer to the source the better IMHO

Thanks - we've since merged the comments hither, since this submitter had the original source.

Submitters: "Please submit the original source. If a post reports on something found on another site, submit the latter." - https://news.ycombinator.com/newsguidelines.html

(the parent was originally posted to https://news.ycombinator.com/item?id=37753079)

> identify high-crime areas that to target for patrolling, which doesn't seem like a huge problem to me

The wonderful Weapons of Math Destruction has a chapter on how this leads to a self reinforcing loop.

More crimes means more police presence. More police presence means more recorded crimes. More crime data means more police …

If only we had statistical tools to compensate for this...

What statistical tools compensate for the generally racist and shitty nature of US police officers?

I don't expect to read this level of asinine on HN.

There are limits to how far stats can compensate for missing data, especially non-random missing data.

I hate how wet streets cause rain.

Yes if you choose to use your data like an idiot then you will generate idiotic conclusions.

This does nothing to negate the value of data used intelligently.

This is a description of how it is currently being used in practice.

Yes you can use data well. It is unfortunately not being used well.

> More police presence means more recorded crimes.

The article in the headline implies the opposite: statistical inference of future crime location -> more cops in said location -> no crime in said location.

It should be fairly easy to back out an out of sample data set for this - ie predicted crime location where the extra cops didn't get deployed and see if that lines up with empirical observations.

If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated. The know it doesn't work.

And it looks like they had some data:

>We examined 23,631 predictions generated by Geolitica between Feb. 25 to Dec. 18, 2018 for the Plainfield Police Department (PD). Each prediction we analyzed from the company’s algorithm indicated that one type of crime was likely to occur in a location not patrolled by Plainfield PD. In the end, the success rate was less than half a percent. Fewer than 100 of the predictions lined up with a crime in the predicted category, that was also later reported to police.

>If it was good at predicting crime I highly doubt only 2/38 departments would have cooperated.

That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more. So it's still what I would expect even if it was reasonably good.

I wish the article would have provided more details about why the PDs chose not to use it. Was it because it was bad at predictions? Cumbersome to use? Glitchy?

It reminds me of a project I was involved with that used a "real-time" computational fluid dynamics model to optimize datacenter air-conditioning. Management was big on the hype, but if you paid attention, the system was routinely unplugged from the actual control system because the facility engineers just found it too difficult to work with.

> That's certainly low, but I suspect (absent some legal framework like FOIA) the default position of police departments is to share less information, not more.

Why on Earth would we want people who have a poor grasp of the law, paired with the means and the license to arrest and kill be transparent about their work..?

We can both agree that transparency is the ideal and still recognize that’s not how incentives are aligned. In general, there are far more downsides and risks to PDs regarding open sharing of information so they generally should be expected to act accordingly.

E.g., I’d like to live in a works where lawyers are concerned with justice rather than “winning” but I’m not shocked when they act like winning is the goal.

To be fair, the way legal system works means if a lawyer isn’t trying to ‘win’ your case, they’re not being a good lawyer - the assumption your client is innocent and you need to defend them is kinda fundamental.

How morality informs the law and the ways lawyers should behave (i.e, should they phone it in/decline the case in if they disagree ethically) is interesting but more difficult as a topic

I think this is mostly correct. Not only are lawyers who aren't trying to "win" not good lawyers, they are possibly breaching their oath. The lawyer has an obligation to try and win, but only after they have done a preliminary investigation before filing the case. There are also some guardrails; your lawyer can't lie about facts they've found out during that investigation, for example. If they do, they can be sanctioned for it. So they can't just try and win at all costs.

With all that said, I think the original point still stands: in most people's minds, an 'ideal' system is one where all the lawyers are trying to seek justice. We just create a less-than-ideal adversarial system as a weak proxy for that given that human nature tends to make the ideal an unreliable expectation. I don't think that's fundamentally different from the PD transparency issue.

"Winning" as the goal is the only way lawyers can be truly objective. Otherwise the justice system turns into a popularity contest. The problem is that any single person/lawyer might ~think~ they know who is guilty or innocent, but they may very well be wrong.

If a defense lawyer knows that his client is guilty, and successfully defends him anyways, that means there was not sufficient evidence to convict. Don't blame the defense lawyer for doing his job rigorously. Blame the prosecution for failing at theirs.

>If a defense lawyer knows that his client is guilty, and successfully defends him anyways, that means there was not sufficient evidence to convict.

This assumes the jury is rational and objective. There's plenty of behavioral psychology that goes against this, but a more glaring observation is that lawyers do not seek to maximize the objectivity of the jury during selection. It's usually very much the opposite.

The goal of the district/trial court system is finding of fact. That's objective. Winning is about swaying the jury, which pulls on the levers of subjectivity. The system is, in part, designed to protect the rights of the innocent. Take the Blackstone ratio, which assumes it is better to let 10 guilty people go than convict 1 innocent. Verdicts cannot be purely objective in a system designed to be asymmetrical.

It seems to me that the incentives of police departments should be irrelevant, and that an organization with that much responsibility should not be able to simply refuse to give out at least certain types of information.

Ignoring incentives takes us into a fairy-tale. Human behavior is largely governed by incentives.

That’s partially my point. Ideally, yes, humans should act impartially and irrespective of incentive. That’s not the world we live in, though.

I think you're missing my point. I'm saying they shouldn't be allowed to give their incentives higher priority. They should be more tightly scrutinized so that their incentives don't matter.

> Gizmodo is reporting on somebody else's investigation

This is Gizmodo's business model. It is a glorified blog. They don't do original reporting. Gizmodo was originally a property of Gawker, and thus the model is essentially commentary and opinion on the reporting generated by other organizations.

Because Gizmodo is really just an opinion/editorial blog, it doesn't really attempt to provide unbiased or fact driven reporting. So it is then no surprise that the actual facts align quite poorly with the headline.

We've since moved the comments in to a different thread. (see https://news.ycombinator.com/item?id=37756211)