Hacker News Reader

Comment by lovecg | Hacker News Reader

You’re assuming a lot about the implementation. In fact I strongly doubt that somebody at Apple literally put “asian” as a blanket filter. It’s more likely an opaque machine learning feature gone awry.

View on HN

Then it's on Apple for not validating and doing checks on the output of their ML models. Apple isn't some scrappy 5 person startup.

How would you comprehensively test for stuff like this?

Take list of top 1000 search terms and run by human reviewers. Take list of top 1000 search terms that are flagged and run by human reviewers. Take list of 1000 sensitive terms (pay a few interns to do come up with them) and run by a human reviewer. Then review top false positives and random false positives.

Apple probably spent more money on the UI components for this feature than all of that would have cost.

A company doing a quarter of a trillion in revenue might be able to figure it out.

Aren’t we supposed to be an industry that prides ourselves on tackling hard problems? We can’t build high tech pattern recognition systems and then just throw our hands up when there’s a false positive.

Does "asian" sound like an obscure term that needs "comprehensive" testing to discover?

Plus, how about partial rollout?

Well, having a list of a few million valid & invalid pages as a test case would probably help.

I did my own test to see and I could find one search term including the word Asian which was allowed (the search was "The History of Asian Nations," which is a book title). I think you're right, it must be more complex than a blocklist.

I tried over a hundred other terms (such as "Asian cuisine" and "college with best Asian American studies program") and none of them were allowed.

You can get out of anything if you put machine learning in between the developers and customers

Machine learning is almost magical in that way: complicated, contextual evaluations of things can be delegated to a black box, and since there's no longer a human people can get angry at for a contested decision, emotions aren't as high and the company can just shrug its shoulders at that silly little computer. It's a cheap way to abdicate moral and social responsibility.

It's one reason that AI ethics is a broader field than people worrying about the existential risk of AGI breakout.

And what's extra depressing is that they're usually not lying. For nontrivial AI, the developers can neither sufficiently answer why it does x or why it doesn't do y.

They're working on it: https://venturebeat.com/2020/01/30/google-researchers-releas...

It's worth pointing out that, post-firing, the present tense doesn't really apply to Timnit Gebru...

If the "machine learning feature" is acting as if it's a simple word-based filter, it's a simple word-based filter.

I wouldn't be surprised if they trained a model on text scraped from somewhere like pornhub and this was the result. It's only surprising they didn't accidentally catch more ethnic terms.

It seems to block "ebony" as well, which is often used to refer to black people in adult content.

Does it also block "teen" and "lesbian"?

It does in fact block "teen", but not "lesbian".

I wonder why?

Because it's 2021. In 1990 or even 2000 it would have also blocked lesbian.

I'm implying that Apple likely looked for that term specifically and unblocked it, while they didn't care to check for the term Asian. But I wonder why?

I implied the same, and also kind of gave away the reason: because it's trendy for companies to pretend to care for LGBTQ now, and censoring it in 2021 would be a faux pas (Yes, Cook is himself gay, but that's not it, because he never was one for causes. Despite being ultra-powerful and rich, he only "went for the cause" for himself and Apple after it became Disney-safe to do so and every company did it - unlike the gay pioneers of the 60s to 90s who did it under really hostile environments).

Reply on news.ycombinator.com