Hacker News Reader

Comment by cantrevealname | Hacker News Reader

Being pegged as a fraud risk just because the user is concerned about privacy rubs me the wrong way. Even if it is entirely legal and the statistical model is valid.

Some years back, some law enforcement agency in the US (but I forgot which) produced a highly-reliable indicator of drug mules on highways. The indicator was black male, driving a late-model sedan, and traveling at exactly the speed limit (when everyone else was traveling 10-15 mph over the limit). The police were stopping and searching anyone who fit these indicators.

Putting aside the questions of racial profiling and whether the statistical model was correct, do you think this is a good idea? I can't articulate my feelings exactly, but I feel it's wrong.

View on HN

I think what you are failing to articulate is the hidden effects of applying this model. Sam Harris kind of makes this point in a clearer way : https://youtu.be/JnYwgcgBYeo?t=2124.

If black people can be stripped and searched more often, sent to jail for less evidence etc the effect will be that they have no reason to try and be successful and productive members of the society. Even if there is no equality, it is essential that there is at least a perception of equality and a noticeable striving for equality. It gives people hope.

I think it's quite a stretch that being slightly more likely to get pulled over would make you an unproductive member of society. I mean hypothetically if they discovered that green cars correlate with drug muling, I don't think this policy would push owners of green cars to crime, regardless of their race.

One factor is easier to change than the other. The car owner can change cars or repaint it. A black man can't trivially change skin color, and they're going to feel that making an effort to be productive is less worth the effort.

If a significant number of additional stops which they have no control over would impact their job performance and reputation (frequently being late, etc), risking getting them fired at no fault of their own, what are they going to do?

> slightly more likely to get pulled over

I don't know why you make this assumption. If the percentage of black people were low, and the number of cops high, that ratio could make the odds of being pulled over arbitrarily high - making certain locations "no-go" areas for black people. I assume as the probability approached 100%, they would just install a checkpoint, or start whitelisting the cars of confirmed good blacks with some sort of sticker or radio ID (just to save on resources.)

It's a good idea if you want to catch drug mules, but a bad idea if you want a fair, free society.

By racially profiling, it easier for non stereotypical mules to get through.

It's also a bad idea if you want to catch drug mules. It's a feel-good idea for the people not being targeted, and an endless nightmare for black men who live there who drive late-model sedans - whose best option really is to move to a less racist area. It's only going to get worse.

>"Some years back, some law enforcement agency in the US (but I forgot which) produced a highly-reliable indicator of drug mules on highways. The indicator was black male, driving a late-model sedan, and traveling at exactly the speed limit (when everyone else was traveling 10-15 mph over the limit). The police were stopping and searching anyone who fit these indicators."

I don't see anything with that sort of indicator. Call it profiling if you will, but I believe it's perfectly harmless assuming no excessive/unwarranted violence is used. That second part is important and drives the point of what I think actually needs to be addressed from the LE side. If it is indeed a bad or poor-performing indicator of a "drug mule", then it will go away eventually.

The problem with that type of indicator is that it will target the same people again and again. An X-ray scan is safe enough to be used in routine medical visits, yet the technician must step out of the room due to dosage effects; so too the effects of even well-regulated police attention are cumulative. (Part of this is probabilistic: if every encounter has an x% chance to go badly...)

And that's true even if the indicator objectively performs relatively well, as long as there are innocent people affected. To go back to the current topic, banning all connections from certain countries is an effective way to cut down on fraud without decreasing sales too much; but that still screws over legitimate users from those countries.

Drug war laws have historically been a proxy for racism and prejudice.

It could be extrapolated from that same data that 'black males obeying speed limits' are just easier to obtain convictions on - after all one has not 'caught' a 'drug mule' until the alleged is convicted.

Studies have long shown it is easier to obtain convictions of black males, especially for drugs.

More importantly for LEO funding today: is money seized can be kept with less blowback if disadvantaged minorities are targeted.

Institutional & societal racism could perhaps better explain the statistic.

Competent criminals adapt their muling system, throwing a patsy to cops once in a while - 'everybody' gets some. Clearly the drugs still get through.

If demand is given, then restricting supply a bit will drive up prices---encouraging innovations to increase supply.

Aside from the probabilistic harm mentioned by sibling posts, there are also issues of self-fulfilling prophecies coming from fishing expeditions.

Assume:

- LEOs use profiling (based on statistical models) to target individuals who are not the subject of investigation but rather subjected to random stop/search/frisk.

- They catch some "bad guys" by following this model.

These assumptions are not dissimilar from the premise outlined in the grandparent post.

Now, let's say the statistical model throws up "20-25 year old preppie white guy with glasses, called Peter" either through error or some odd outlier stat. If LEOs start to check out every 20-25 year old matching that description, sooner or later they're going to find drugs or a gun or a lot of cash or something. Their statistical model gets updated with these stats, and before long it becomes a truism that a disproportionate number of preppie Peters are criminals.

If you shine a spotlight somewhere, that's where you'll find what you're looking for.

My feeling is that it's ok to make judgements on the population and act on that, it's not ok to use judgments on the population to negatively affect your treatment of the individual.

This is easier to explain with gender differences: It's ok for a company to decide to make financial considerations based on the number of women they employ and chances of maternity leave. It's not ok to not hire an individual based on the fact that they're female and therefore have a chance of needing maternity leave.

It's a valid conclusion that on average men run faster than women, but you should assume that in any given male/female pair the man will run faster.

It's difficult to know where to set the threshold for 'negatively affect' though.

For instance, black men have a higher incidence of heart problems, and different medical decisions need to be taken in that case due a different risk balance. This (to me) is clearly just good evidence based medicine.

In this case I'd argue the problem is that 'black' is a signal used in a lot of models, so discriminating on that becomes a large cumulative negative effect so it shouldn't be used. If 'red hair' was a signal in some situation, then that would be fine because people don't normally get affected by that. The rare situation of being stopped incorrectly is outweighed by the benefit.

If that is wrong, what is right?

Should the police instead do less efficient completely random searches? Should the taxpayers hire more police officers to make up for this loss in efficiency?

Should businesses accept losses from fraud to placate your feelings?

While it may not feel very good to be sending signals that associate you with crime, that is a cost you have to accept of sending those signals.

In the case of police searches, I think attention would be better spent on making them more humane, rather than less racist.

It's wrong when those signals come from birth.

I feel there is something deeply wrong with this sort of over-statistically/analytically focused thought chain, allow me to try to elaborate.

Many signals have absolutely nothing to do with wrongdoing whatsoever. Say that hypothetically every drug dealer makes large cash deposits, and this is marked in a database somewhere as a "very reliable indicator". Well what about taxi drivers? Street Vendors? other occupations that deal almost exclusively in cash, should they also be suspected?

What happens if somebody otherwise innocent of anything in particular triggers multiple "flags" like this, do they deserve to be discriminated against?

What if the flags are stuff like being black, being poor, having a friend that has been charged with a crime. What about living in the "wrong" neighbourhood, how about the wrong genetic markers. How about a family history of mental illness, what about if you dont own a cellphone? What if they are angry about the government and have publicly protested? What if you accidentally like ISIS on facebook, or your friend does it as a joke? are these acceptable "black flags" too?

What about if one of your close friends trips one or more of these markers? What if several do? Are you just as untrustworthy?

It gets much choppier when you introduce something like neural net learning into the equation. It will form links between activity that are basically impossible to decipher by a human at all, should we accept these utterly opaque judgments because "on average they are pretty reliable"?, what about the cases where they are 100% dead wrong, should they just be considered collateral damage?

A huge class of people could end up going through life denied basic services like a bank account because of things ENTIRELY beyond their control, or even understanding, all they know is they managed to piss off some equation somewhere.

It gets better, because on average people with bad scores on one system "reliably indicate" they will be bad faith in other contexts too...

Lots of businesses will accept this as an acceptable cost to keep fraud rates/whatever down, I don't find it acceptable, its almost the definition of injustice, and has a grave potential to create a vast underclass of "poor social credit score" people (who being a member of, even by association will assuredly be another hit against your scores)

How happy would you be, if via no fault of your own you were deemed an "unacceptable risk"?

So yes, it is inhuman, and wrong. I guess that -100 Social Credits for me.

Intentionally or not, you seem to have done a solid job of describing existing AML/KYC legal requirements.

Example - http://www.acamstoday.org/domestic-high-risk-geographies-eme...

https://www.fincen.gov/law_enforcement/hifca/

Agreed. This mentality doesn't only lead to being denied a bank account; it leads to murder, literally.[1]

How is this any different from any number of discriminatory practices in the past, which are now (at least superficially) illegal, such as "red-lining" black neighborhoods to deny home loans?

‘We Kill People Based on Metadata’ [1] http://www.nybooks.com/daily/2014/05/10/we-kill-people-based...

The scariest thing is my mind is the potential for this to become self-reinforcing. Marginal people are marginalized even further, and forced to turn to crime to survive, thus "justifying" it even more.

Also, lots of chilling effects: you can't decide to be black, but you can decide whether to pay in cash, or drive the speed limit.

This new years, I made a large liquor store purchase (about $600) in Manhattan, not in my normal geographical area. The transaction was declined, but all I had to do was to answer "yes" a text that I got from the issuing bank, and then swipe again. Was I "profiled"? Yes. Did Visa do a good job protecting me and itself with my profiling? Yes. Try running a real business with this purist approach. You will exceed 1% chargeback threshold in the first year, and your payment processor will either shut you down or will require a huge collateral.

Yeah, I think this is an appropriate way to do profiling. The declined transaction was explicitly tied to something you did, so you understood why it happened. The bank also provided an easy way out.

Where profiling becomes dangerous is when it's tied to something you are or something you did in the past: it can be harder to identify why you're being profiled in these cases. Most of these cases aren't resolved through a single text, either. Profiling is a useful tool, but we have to be careful to do it in a humane way.

We actually already do accept less police efficiency in favor of personal liberty in many areas. This is an American tradition.

E.g. The Fourth Amendment warrant requirement. No one doubts that we'd find more criminals and contraband if police could make impromptu home searches of anyone at any time. But that's how a police state functions.

And we've decided to exchange efficiency in crime reduction for civil rights.

Are you sure? Do countries without a 4th amendment have more criminals in jail and lower crime rates than we do?

America has the most `criminals' in jail. America likes to lock people up.

(You'd have to find a different metric to figure out what effect the 4th amendment or similar equivalent in other countries has.)

I think your missing his point. He was assuming that the cops were already doing unjustified random searches of every car on the road.

Assuming they are already doing this, wouldn't it be preferable if they switch to a model that decreases the number of innocent people that are searched by 10x? Wouldn't that be a net good to society?

Of course if your opinion is that they shouldn't be allowed to do random searches at all, this whole thread is really irrelevant.

>If that is wrong, what is right?

easy. ignoring protected classes [1] - thereby losing part, but not all of your signal. i.e. don't see race.

[1] https://en.wikipedia.org/wiki/Protected_class

Security profiling doesn't work, as bad guys adapt very fast.

Probably quite a lot of drugs are moved by white guys in shiny rented cars, because those are less likely to be profiled.

>Should the police instead do less efficient completely random searches? Should the taxpayers hire more police officers to make up for this loss in efficiency?

Jesus, dude. Obviously they should.

Who the fuck cares about efficiency if they're pulling over people because they're black... in sedans.

You need to calculate the base rate. If only 0.1% of cars are drug mules, then that statistical test might decrease the number of false positives by 10x to 1%. But it still means you will search 99 innocent people's cars to catch a single drug mule. Which is well below probable cause, and probably a net harm to society.

If they could predict which cars were drug mules with 30% accuracy that would be acceptable I think.

The worst part of the model is that it is self-reinforcing. The characteristics that are being selected for have no connection to the thing being searched for other than previous correlations. Focusing investigation in that area will make the correlation even stronger. For example: black people in the US are less likely to do drugs than whites, but black people are imprisoned for drugs at six times the rate of whites.

-----

"The race issue isn’t just that the judge is going, ‘Oh, black man, I’m gonna sentence you higher,’” she said. “The police go into low-income minority neighborhoods and that’s where they make most of their drug arrests. If they arrest you, now you have a ‘prior,’ so if you plead or get arrested again, you’re gonna have a higher sentence. There’s a kind of cumulative effect.”

http://www.huffingtonpost.com/2013/09/17/racial-disparity-dr...

> Putting aside ... whether the statistical model was correct

You can't put that aside. If the model is simply flawed then following it can't be a good idea. If the model is perfectly true and accurate then not following it is a bad idea. Of course nothing is perfect, so it would never be as black/white as that, but I don't see how any discussion that ignores the model is at all pointful.

> the questions of racial profiling

As with any statistical modelling this can get pretty hairy and potentially has recursive effects that are self-fulfilling. If you search just black people because the model says more of them get caught carrying drugs, you risk perpetuating the statistic whether it is correct or not because if you are hardly ever searching white people few are going to get cause so it will remain that more black people are found to be carrying. Unfortunately good statistical modelling is damn hard and criminal profiling is far from the only place you'll find these problems (many a scientific study has been discredited or otherwise written off some time later because flaws in the modelling and measuring are noticed in subsequent analysis).

Sampling bias.

If you're searching (and targeting for search) a specific set of criteria, then that set of criteria will be what indicate high probability.

As a friend pointed out long ago, it's why you don't ask a group at a scheduled meeting what their preferred meeting time is. You're already selecting for those preferring the current time, and excluding those who find it unappealing.

As long as it is just a quick search, what's wrong with it? I'm confised by US views on this: I go through pat down search every time I enter club or a mall, I'm quite happy to comply and never have any problems with it. I don't own the car, so I don't know how inconvenient and long a search can be, but if the police are aware that this factors are weak and give a lot of false positives, I don't think they treat every fitting individual as a deug mule.

It's the distinction between what's optimal for the user vs. the company.

It's completely reasonable to not like being profiled based on arbitrary features. The question is: what alternatives are there to preventing fraud?

Privacy concerns are an opinion. You aren't born with them.

Why is racial profiling wrong?

Aren't you just making statistical inference based on past experience?

if that is wrong then all learning is wrong.

"Aren't you just making statistical inference based on past experience?"

Yes, but if your behavior in the past was also biased, going off of arrest records or something like that won't get you better / less biased results.

Racial profiling works to some degree, but it means that innocent people who match the profile have to put up with a lot more intrusion into their lives. It's similar to the moral question about where to draw the line: would you rather imprison an innocent, or let a guilty person go free?

If you're from the U.S. you seem to be forgetting some very hard lessons that were paid with blood in the past. There is a reason why we have a bill of rights. There is a reason for our rule of law, of requiring warrants before searches. That our 'forefathers' said that it's better that 10 guilty men go free than one innocent man be falsely charged. Racial profiling is wrong because you're infringing on the freedoms of innocent people that have committed no crimes.

Reply on news.ycombinator.com