Hacker News Reader

Comment by crazygringo | Hacker News Reader

The problem seems less about racist bias specifically and more about unbelievably dumb tags generally in the first place.

"Stunner"? "Looker"? "Mantrap"? Or even trying to tag people's images with categories like Buddhist, grinner, or microeconomist?

What were they thinking?! Clearly these tags were never curated in any remotely responsible way -- for quality, for sensitivity, or just usefulness at all -- and I'm shocked they were ever intended for academic or research use. No wonder image recognition gets a bad rap, with input data like this.

View on HN

There are ~32.000 tags. No surveillance system is using ImageNet tags to classify people into Buddhists or Not-Buddhist. Most researchers ignore these tags and focus on a 1000 classes, and know that 32k performance is not good (and these artists have no intention of making it work at all). What they are uniquely trying with this Art Project is as much research as it is activism. Note that "mantrap" is defined in synset as "A trap for catching trespassers", and that you are bound to find weird stuff among over 30k categories (imagine what you can say with the 32% most popular words in French...).

This is a photo in question: https://memepedia.ru/wp-content/uploads/2019/09/imagenet-1.p...

This is the route the network took:

person, individual, someone, somebody, mortal, soul (6978) > female, female person (150) > woman, adult female (129) > smasher, stunner, knockout, beauty, ravisher, sweetheart, peach, lulu, looker, mantrap, dish (0)

So it was (politically) correct on the first three categories, and the last one was either a crapshoot (and she could also have gotten to the subcategory of "prostitute" > "streetwalker, street girl, hooker, hustler, floozy, floozie, slattern") or she really is posing in a common "beautiful woman"-way. (The global description for this route is "A very attractive or seductive looking woman" and often triggers for females with tilted heads and lip curls).

You can turn any faces dataset into a labeled face color dataset, so if a black person being subclassified as "negro" is problematic bias or encoded racism, then all such datasets are suspect. Noisy labeled data is the norm, not some horrible exception to be avoided at all costs.

The fact that we have a machine learning algorithm that labels _humans_ as smashers, prostitutes, or convicts is a problem, irrespective of whatever technical justification can be made for it.

But who artificially created that problem? The artists. There is no marketing company that has intelligent billboards scanning the public for prostitutes. There are no researchers seriously using CV to classify convicts (at least not in the West, and not with ImageNet). That could be a malicious usage problem. This is either a non-problem or Armaggedon for all ML CV datasets, because you certainly can use most CV datasets to train a crappy classifier to output offensive labels. If I train a people photo tagger using a dataset used for combating poaching monkeys in Africa, then who is at fault? Certainly not the researchers who published that dataset with the idea that the data would be used with common sense and scientific rigor, not adversarially -- to make a political point attacking the very existence of that data. The "exposed" bias is trivial.

It is the technical justification that should be all that matters for a canonical academic dataset. Science does its best to be apolitical, but then politics ("red bull drinking white men train racist and sexist classifiers") is forced upon it, and we can't really have a productive conversation about bias and ethics anymore.

AI needs common sense knowledge of the world to improve. Censorship so science does not offend our sensibilities, would only make it so Google Image Search (a machine learning algorithm) does not return any images of people when you search for "prostitute". Heck, the AI would never learn the difference between a male and a female prostitute. Destruction of accessible knowledge so we (aka: people on Twitter who think AI is the terminator, or the director of the internet) don't get offended by some primitive ML model-as-art-project forced to make errors or awkward classifications. That's a sentence the academic ML community could do entirely without. No benchmarks or duckface selfie would be hurt. No unfortunate third-world souls hired to scan 20 million + internet crawled images for wrongthink, only for the machine to do the unsupervised learning in a hug box, not a black box. Oi mate, you got a loicense fer that label?

Just wait until the activists find out who wrote the first 100 8's added to MNIST. Nobody but MIT would be associated with her, if they found out what she did.

> What were they thinking?! [...]

In addition to their statement, linked in a thread below [0], what they were thinking is pretty out in the open in the abstract of the original paper:

"We introduce here a new database called “ImageNet”, a largescale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. " [1]

That paper is now 10 years old, WordNet itself had it's first release somewhere in the 1980s. A lot happens in research a decade down the road and lots of research towards biases of datasets are relatively recent in that sense. It can hardly be blamed on the researchers that people use their datasets in production merely because "they're large and there in the open". As their statement indicates they're actively working on the issues that arose during the last few years.

Most research datasets are fit for a very specific purpose, to me the larger problem here is two-fold: "AI" fearmongering on one side and startups/corporations on the other side lacking the necessary skills to select and filter their training data. Hopefully that'll be a matter of the past once the hyped data science field reaches maturity.

edit: Wordnet seems to include said terms to this day, [2] for example still lists "mantrap" associated to "beaty" in the synset for "a very attractive or seductive looking woman". Although I'm unsure how they should handle these cases, both are words and that's one definition used in practice, even if we find it reprehensible. They seem to have removed the n-word but that's about the only removed instance I can find in a cursory search of sensitive or insulting English words. Maybe they should clarify sensitive annotations like they do with vulgarity.

[0] https://news.ycombinator.com/item?id=21054770

[1] http://www.image-net.org/papers/imagenet_cvpr09.pdf

[2] http://wordnetweb.princeton.edu/perl/webwn?s=beauty

But removing the images from ImageNet, an inanimate set of images that cannot make value judgements, will surely fix this problem...

Our culture is broken and nobody is willing to stand up to insanity or even admit to it. People are too happy to do completely insane and senseless things if it saves them from the social lynchings.

Not sure about "mantrap", but I could imagine "stunner" and "looker" in the pages of a magazine article - A red-carpet celeb photo "Jane doe looking stunning! And with her hubbie - what a looker!".

I guess the problem with things like "microeconomist" are that other tags such as "fireman" might be valid (if a picture contained clues about vocation, for example) - furthermore, rather than condemn the label, we might instead consider not demonising a neural net, and taking its results at face value; in this case an indication that most labelled micro-economists in the dataset are suited white men.

Buddhist seems pretty easy to categorize? If you see someone standing in front of a buddhist temple wearing an Kāṣāya (had to google the actual word: the orange robes) they're pretty likely to be a buddhist, right?

Agreed. These are not descriptive labels you can infer from an image.

They may be, however, labels generated by users from a context.

Reply on news.ycombinator.com