Computer personality judgments more accurate than those made by humans: study

They say their models are more accurate, but more accurate with what data? It turns out models are better than humans at predicting how people rate themselves on a personality model. So humans are rating their opinion, vs algorithms that are predicting what people would say about themselves.

So I wouldn't take those numbers at face value.

The external validity comparisons are more compelling, but still, its telling that self-reported data is better at predicting self-reported events.

It's entirely possible that computer based models are better at knowing us than humans, particularly since algorithms can pay far closer attention to us, but there's plenty of literature showing that we have the capacity for self-deception, so I'm not sure how good a ground truth you can have here.

The fact that this is even close (let alone higher for computers) is pretty incredible. It is also quite scary for anyone with a public profile and speaks to the potential power that an organization with bad intentions or ulterior motives could have (ex: Cambridge Analytica).

Authors of this paper communicated with the people behind the information operations campaign [0, 1], but it's not clear to me how closely they collaborated.

[0] https://motherboard.vice.com/en_us/article/mg9vvn/how-our-li...

[1] https://www.theguardian.com/news/2018/mar/17/data-war-whistl...

I had noticed with with Netflix. After telling it enough movies I liked (over say 100) it pretty much figured me out and it knew me better than I knew myself. In other words, I'd see a movie description, watch the trailer and think "Nah, I wouldn't like it". But Netflix was telling me I'd like it, quite often after I watched it I did end up liking it.

On another note, I always tried to steer away from taking personality tests. I have a strange belief that once I know the "official" results it would somehow restrict my what I would attempt later in life. I'd think "Oh well your personality officially doesn't match, you shouldn't try that".

If they want to stick to the questionnaire, then train the computer model on the questionnaire too and let's see who wins. Giving one side more powerful data and giving the control just a questionnaire is like giving a robot a machine gun and a human a knife and being surprised who wins.

But nevermind what can be predicted from the likes. They say that coworkers have r=0.27, while spouse has r=0.58. This looks like a serious problem with the test more than anything else.

This may be interpreted as "cowokers don't know you (and don't care...)" kind of stuff. But coworkers often have a comparable amount of contact with a person vs spouse. Alternatively, one could also interpret this as there is no "the one true personality", but different personalities in different environments (at least when we define personality as a result of a test). And so coworkers are not wrong, they just see a different picture.

And there might be other possibilities.

In fact, one should try to predict the coworker's (and spuse's, ect...) scores of a person from person's likes, in addition to trying to predict person's own assessment.

> (...) the accuracy of the personality judgment depends on the availability and the amount of the relevant behavioral information, along with the judges’ ability to detect and use it correctly (1, 2, 5). Such conceptualization reveals a couple of major advantages that computers have over humans. First, computers have the capacity to store a tremendous amount of information, which is difficult for humans to retain and access. Second, the way computers use information—through statistical modeling—generates consistent algorithms that optimize the judgmental accuracy, whereas humans are affected by various motivational biases (27). Nevertheless, human perceptions have the advantage of being flexible and able to capture many subconscious cues unavailable to machines. Because the Big Five personality traits only represent some aspects of human personality, human judgments might still be better at describing other traits that require subtle cognition or that are less evident in digital behavior.

This last sentence is important: Looking at scores from quantitative psychological models, it is easy to forget that although a model has some predictive ability, the thought of a _complete_ model is an illusion. For example, low and high IQ scores have some merit in predicting people's success in life, but anyone who knows several people claiming to have high IQs will observe that such people's ability to actually accomplish something with their claimed ability varies greatly.

I don't doubt that psychological models can be an efficient way of filtering through many potential candidates for a position, deciding what ad to show someone or predicting the chances of someone becoming a drug addict. They could even help us see past some of our cognitive biases when making a decision. But I don't think that we will ever, at least in the next hundred years, reach a point where most people let an algorithm trump their own judgment in decisions where the emotional stakes are high. For example, in choosing a life partner, dating services will probably use such algorithms to filter the candidates, but the user will probably make the final call on who to settle with.

After all, personality tests have been around for decades, but most employers use a good old fashioned face-to-face interview to choose the person to fill the position.