Hacker News Reader

Comment by kakhag | Hacker News Reader

Academic machine learning is a paper mill. Sometimes there is a rare success like the transformer model and everyone converges on that.

I think the comments here are arguing past each other. If a machine learning model can correctly classify $attribute by photos in 70% of the cases, that is an academic result.

Should it used for the ever popular pre-crime detection scams or for misclassifying feminine looking men as homosexuals? No, that is nasty and is forbidden in the EU. At least for companies, the security state of course does whatever it wants without any repercussions.

Should it be used in principle? No, classifying humans by a machine algorithm is dehumanizing.

Should research be done at all? If the subjects are volunteers and it is confined to a university, it should be allowed. But researchers should not be surprised it they are confronted with obvious parallels to phrenology. This literally is phrenology.

View on HN

I just want to push back on Academic Machine Learning is a (low quality with no novelty) paper mill and devaluing researchers efforts.

To be clear ML research has “paper mill” problems but we should be careful that we don’t imply that there are only “rare successes”

There are many many amazing results published at ICLR, NeurIPs, ICML every year that are important developments that are not only research successes but also commercial and open source success stories. For example LoRA and DPO are two recent incredible development and these are not “rare” - this years ICLR had many promising results that will in turn be built on to produce the next “transformer” level development. Without this work there are no transformers.

Even transformers themselves were a contribution whose impact only became valuable through the work of many researches improving the architecture and finding applications (for example LLMs were not a given use case of transformers until additional researches put the work in to develop them)

> Should it be used in principle? No, classifying humans by a machine algorithm is dehumanizing.

I see utility in a machine that can help me notice when I'm having a bad day (or a very bad day ie. when having a stroke) or ... help me figure out what the fuck is going on with my totally hypothetical mental illness.

As a personal assistant it has incredible power, but it should effectively be self hosted.

I see plenty to fear when other people deploy AI to “determine” and judge my mood and my non-hypothetical mental illnesses without my knowledge or consent.

Not unreasonably so! I also like my privacy (or what's left of it) ... but when (if? no, unfortunately most likely when, and maybe even "soon") it goes at least I want to have some benefit for myself.

Arguably the current generation of school-age kids (genZ? genA?) already lost it. Because even if they don't have a phone, others do. (Though hopefully this trend can be reversed. Classic bullying at school is bad enough now with cyberbullying becoming the "norm" things are definitely not looking great.)

> But researchers should not be surprised it they are confronted with obvious parallels to phrenology. This literally is phrenology

Phrenology was classified a pseudoscience not because it could lead to socially bad outcomes but because it didn't work; it had no statistical/empirical grounding. If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience.

To add to your point - you definitely can make predictions about aspects of someone’s personality by looking at their face with the expectation that you’ll do better than chance.

The paper calls out inferring political leanings as an example of pseudoscience. Give me an American’s age, gender, and race (which I can roughly identify by looking) and I’ll tell you, better than chance, whether they’re a trump supporter.

Agreed, but I think the parallel with phrenology should be undestood more along the lines of "providing a justification for unfair decisions".

I remember a case where Amazon used a resume filtering bot that systematically rejected female candidates, because of bias in the training data. So we might go from "you can't be free because your skull is too small" to "you can't get the job because the computer says so".

> Agreed, but I think the parallel with phrenology should be undestood more along the lines of "providing a justification for unfair decisions".

I think we cannot discount the degree to which ideology played a part BOTH in the promotion AND rejection of phrenology. And when the racist ideologies eventually became taboo, anti racist ideology won.

If we consider the actual science, it was probably highly tainted by a desire to show that certain human lineages were superior to others.

But there DOES appear to be a correlation between brain volume and IQ. When controlling for "race", this correlation is typically reported at 0.3-0.4, meaning brain volume accounts for 9-16% of the variance.

However, if we reject "race" as a social construct, and include people of all "races" in our analysis, the correlation goes up to about 0.6, or 36% of the variance [1].

[1] https://emilkirkegaard.dk/en/2023/03/modern-neuroscience-con...

No but you don't understand, the GP finds it icky therefore it's "dehumanizing."

> If it turns out deep learning really can reliably predict things about people's personality from their faces, that doesn't make it a pseudoscience.

What makes it pseudoscience is that it's not theory-driven. These are statistical models that recapitulate distributions in their training data. It's Brian-Wansink-style p-hacking [1] at a massive scale.

[1] https://www.vox.com/science-and-health/2018/9/19/17879102/br...

> What makes it pseudoscience is that it's not theory-driven

Science is primarily empirical, and there's nothing inherently wrong with an effective theory that works, until we find a better, principled theory.

> It's Brian-Wansink-style p-hacking [1] at a massive scale.

Sometimes. Models that generalize are in fact generating theories though.

Reply on news.ycombinator.com