Hacker News Reader

Comment by jawns | Hacker News Reader

The Civil Rights Act requires companies not to discriminate against legally protected classes. That includes both disparate treatment and disparate impact.

I can't see how "personalized pricing" would remain unchallenged, given those hurdles, if it truly means pricing fares based on individual characteristics.

It's totally fine to have loyalty programs that reward people who fly Delta frequently with better rates.

But when an ML model is making inferences about a person's willingness to pay based on what data a company has collected about them, that really feels like it's moving into murky waters, even apart from the predatory element.

View on HN

There's complexity here that could sway a court. A few years ago, we engaged an AI company to identify potentially problematic hospitality customers. Those were required to check in in person at the front desk instead of the app. Sounds like it could fail those tests, but those tests are for humans and this is AI!

the second it leaks that "potentially problematic hospitality customers" disproportionally target a minority you and the AI company will be sued aggressively from all sides.

Oh sure, and the publicity would actually be worse. But these were organized crime gangs that would check in, steal everything, and leave in 15 minutes. So the investigation and targeting was fairly narrow. It wasn't designed to filter unruly guests or anything else.

Maybe their excuse will be that it can't be proven whether or not the deep learning model is discriminating based on this or that, it's just taking in as much data as possible?

You’d have to demonstrate that the pricing consistently is high/low based on one of the protected classes to successfully challenge it.

Assuming their model is accurate at predicting economic value and assuming that the wage gap between men and women still exists, wouldn't it be relatively straightforward to show men as a group are quoted higher prices?

Men aren’t a protected class. I’m not sarcastic: They really are not, it’s legally ok to charge men more.

The general problem here is that "disparate impact" is incoherent. If you charge everyone the same price then you're de facto excluding people with less money, and therefore have a disparate impact on the basis of various protected classes because of the correlation between e.g. race and poverty.

Now suppose you charge less to people with less money. Is that what you're supposed to do? No, sorry, there was a preexisting disparity in your customer population on the basis of national origin and doing that makes it worse, you still lose.

There are multiple protected classes and how something affects one of them is often the inverse of how it affects a different one, so it isn't possible to not have a disparate impact. And then whether something is considered to in any given case is nothing but politics and vibes.

If you are getting a different treatment purely because you’re a man, then you can absolutely sue and win.

"Men" and "being a man" sure as hell is a protected class, and there are plenty of lawsuits to back it up. e.g.

https://www.foxnews.com/story/texas-man-settles-discriminati...

like it's literally a 5 second search on the google or bing to prove

I thought the bar was that you had to prove that it doesn't discriminate which a is much higher, I thought this is what drove many settlements.

Personally I would rather limit all price discriminations to a simple predictable and publicly inspectable customizable formulae. It's like the finance guys who exalt the virtues of increased liquidity but that disappears the moment you try to use it. Similar for the extra complexity for price discovery.

As a student of political science, I agree with you. However, the current political administration and Supreme Court seem unlikely to be interested in protecting consumers in this way.

the current administration and SCOTUS are doing everything they can, aggressively, to screw over the average consumer

You’re assuming that we have a functional government that doesn’t turn a blind eye to discrimination.

yeah seems likely if they know peoples browsing habits they could easily indirectly be basing prices on protected characteristics

Interesting thought.

What specific protected class is "harmed" by personalized pricing?

And how would you prove that the model discriminated against them based on a protected element?

All of them as they’re being used to compete and discriminate against each other.

That's not really how it works in the US. You have to prove that you were dicriminated against based on a protected class.

Everyone getting different prices doesn't mean its illegal:)

You can’t charge black people systemically different prices than white people, for example. Neither better nor worse prices. Proving causality would likely be the hard part, but a systematic survey of disparate pricing could show disparate impact. AI models frequently are invisibly biased on race because of some feature that operates as a proxy measure that the developers don’t recognize. eg something as simple as ZIP code combined with a name is a highly reliable predictor of race, and there are many similar, far more subtle ways bias can creep into a model.

How can anyone allege discrimination when all you have is an AI telling you the price you get? It’s not open source and you don’t get to see data across all races or sexes or whatever to spot a pattern.

I think we just need new regulations and laws. Maybe make income or wealth a protected trait you cannot discriminate on. Or require transparency and reporting on algorithmic pricing. Or higher taxes for businesses that use these tactics.

But that’s years away, if ever. I bet all airlines will do this. I’ll just fly even less

> “make income or wealth a protected trait you cannot discriminate on”

Seems like that would be problematic for progressive tax rates.

Honestly it’s pretty clever. “Wealth is a protected class” would fit right in with the modern right-wing rhetoric, continuing the inversion theme already established with “actually whites are the targets of racism” and “actually Christians are persecuted in the West”, etc.

Reply on news.ycombinator.com