The Ongoing Accomplishment of the Big Five | Hacker News Reader

The Ongoing Accomplishment of the Big Five

carcinisation.com

61 pointsbarry-cotter17 comments

The same author's discussion of surveys is even better: https://carcinisation.com/2020/12/11/survey-chicken/

> What is interesting is the degree to which the Five Factor Model is insulated from falsification. It makes no causal predictions; others may make predictions about correlations, and these may replicate, or not.

I remember struggling with feelings I would describe similarly when being taught how to fit an ARMA model to the stock market. Professor was very proud of himself and I was baffled about why he'd thought he'd done something. Eventually I decided maybe such models serve a function in academics very similar to a compression algorithm. The end goal is simply to be able to describe the data without having to present the data line by line - nothing more. No predictive or support in decision making is implied.

The Big 5 model looks like it is a lot better than nothing. "Do you normally feel good?" and "Do you normally feel joyful?" are different questions, but determining when they are practically different requires a deep mastery of the language.

Mr. Banana makes some good criticisms and the model will probably be abused to justify some stupid stuff out in the wild as many things in psychology are. But for academic sense, being replicable is all the model needs to do unless someone comes up with something clearly better.

What I took away from that analysis is that the entire field is self correlated (good) but lacking any underlying falsifiable theory (bad) or any ability to predict anything (bad).

Repeatability, falsifiability, and predictability are the three requirements for any science. Remove one or more and it’s no longer science regardless of how much you want to believe it’s providing useful results.

The history of science is littered with many such approaches. It could be the big five could eventually find better theoretical footing. My observation of the social sciences is less keen on this outcome. They perennially fail to accomplish it and thus move to “greener” pastures in terms of their financial incentives.

For an example, consider the case where the study authors had a hypothesis that three of the five traits would predict something, only 2 did and they decided this was a success. Meanwhile astrophysics is arguing about potential sources of errors when results are off by 0.001%. Obviously different fields but the social sciences lack of establishing a solid quantifiable footing is non-ironically dismissed as “humans are difficult” rather than acknowledging that the basic science that’s attempted to be done is flawed and will never yield scientific results with the course it’s on.

The fundamental problem with social "science" is that it's trying to predict human behavior, but human behavior is uncomputable. Literally.

If you publish a study that says humans will do something, the humans will read your study, take it into account, and then do something different. That's the halting problem, game over.

The human race, to which so many of my readers belong, has been playing at children’s games from the beginning, and will probably do it till the end, which is a nuisance for the few people who grow up. And one of the games to which it is most attached is called “Keep to-morrow dark,” and which is also named (by the rustics in Shropshire, I have no doubt) “Cheat the Prophet.” The players listen very carefully and respectfully to all that the clever men have to say about what is to happen in the next generation. The players then wait until all the clever men are dead, and bury them nicely. They then go and do something else. That is all. For a race of simple tastes, however, it is great fun.

-- G.K. Chesterton, The Napoleon of Notting Hill, 1904

This seems equivalent to the insight that a machine cannot be a programmer, ie., a system which cannot frame open problems cannot design systems which solve closed problems.

NB. open here meaning aprox. "a problem class which is framed as a class by the solver"; closed meaning, "one of a class of pre-solved problems".

Since humans are extreme open-problem framers, their framing strategies are themselves updated adaptively (by the environment) -- which cannot be reduced to any class of closed problems.

Since (social) science does not have a theory of open-framing (ie., intelligence & problem solving proper) each description can only be a closed-solution description of a particular behaviour -- which then becomes invalidated when we reframe (eg., via reading the journal article).

Your claim, re the haulting problem, suggests that there -- in some sense -- can never be a computable theory of open framing (, this seems highly plausible to me). In which case psychology is broadly impossible.

However things like anger, family, etc. do exist. Psychology as a science of behaviour may be impossible; but there is a science of deterministic forces of human behaviour.

The entire mathematical field of game theory describes how any agent (human or otherwise) would behave in various situations with carefully crafted rules. Mathematicians have modeled successfully how herd animals move (including humans) even when though the individual reasons for any given movement are opaque.

Obviously these are still primitive tools but they surely provide a better and more solid foundation.

You’ve also thrown the halting problem phrase around here but unless you have a paper actually establishing a mathematical proof, all you’ve actually done is state a hypothesis. Furthermore, the halting problem only says that it’s impossible to prove a priori whether or not a machine will return a result or run for infinity. By definition the algorithm it’s actually executing is computable. The closest analogy would be more that we won’t know if there is a point in the future where we can accurately model human behavior but we can continue trying and we may well succeed. Remember the halting problem says the process may or may not complete but that we can’t prove it a priori of running the process.

As for your “we’ll change our behavior if we successfully model”, that’s a reductive argument because it presupposes the format the solution would look like. You’re assuming that the modeling would look like “if scenario X do action Y”. Proper modes here would be probabilistic and have specific criteria for the environment and rules.

So more like “patriarchal societies have a probability of X% of forming when environmental pressures are set Y” or “societies are most stable when conditions A, B, C exist”.

If you think human behavior is unpredictable at all by computers, would you care to short any tech advertising company like Google or Facebook? After all you should be able to provide a proof and instantly devalue their CPC overnight whenever you want.

Edit: Also if you truly believe what you said, “social science” needs to be renamed to “social tea leave reading”. The science part is pseudo-science.

Of course human behavior is computable, at least in aggregate. How do you think marketing works? The mind is a computer and there's nothing special about it.

There's a lot of thing special about it, whether it's a computer or not.

For one, it's a whole different architecture.

Second, it's several orders of magnitude more complex.

Third, it changes based on all kind of strange inputs, not just conditions in the program and some IO (e.g. changes in chemicals in the body). Even the substructure where the "program" runs changes properties (like a CPU changing material properties mid-execution).

Fourth, the "program" is self-modifying constantly, as opposed to a static program or different reprogrammings of an FPGI.

Fifth, there's no manual or clear schematics.

The accomplishment is 'limited' in its applicability:

> As a novice to the study of the Big Five, I found that I had many misconceptions. For example, I thought that the Big Five were considered to be universal, in some sense, and not just descriptive of WEIRD college students. But in fact, it is not simply that the Big Five factors fail to fall out of the analysis of large surveys in other languages and cultures. They don’t even fall out of Big Five-specific questions administered in non-WEIRD populations (Gurven et al., 2013). Many of the concepts mentioned in the Big Five survey instruments do not even exist in non-WEIRD languages as such, particularly abstract nouns (Gurven et al., 2013). Of course a concept that does not exist across cultures could not form the basis for a universally important personality trait, as measured by language.

Be mindful of other cultures / worldviews:

* https://en.wikipedia.org/wiki/WEIRD

* https://en.wikipedia.org/wiki/Cross-cultural_psychology

Joseph Henrich recently published a book on the topic, discussed about a month ago:

* https://www.theatlantic.com/magazine/archive/2020/10/joseph-...

* https://news.ycombinator.com/item?id=25227902

This was a fantastic essay on psychological measurement of personality types.

The take-home for me was:

  The Big Five are, in a sense, protected from 
  falsification. They make no predictions; there is no 
  underlying causal model. As I understand it, no study could 
  be devised to prove that the Big Five aren’t real, 
  because they make no formal pretense to reality. They are 
  innocent mathematical constructs that fall out of 
  particular survey instruments administered to particular 
  populations.

It is a data point in the idea that

1. only things that could conceivably be measured are real (unmeasurable/unobservable things are magic / not real)

2. but not all measurable (=real) things are important / valuable

Acts of clicking links are better data than surveys.

And advertisers still cant predict what I may be interested in.

They mix in that deliberatwly, so as not to creep you out...?

>> They mix in that deliberately, so as not to creep you out...?

As a free software advocate and maintainer, I've started promoting Free alternatives to commercial software in the comments under their owns ads that show up in my FB feed. That participation seems to increase the ads. So no, they don't have a clue and keep paying FB to show me that stuff.

This is probably true, especially given the state of the surveys (for example, it required less than 10 minutes perusal to uncover obvious bias against distracted-type ADHD diagnostic surveys and by extension, females with ADHD).

Although online history may provide superior illumination of a patient’s condition, crucially, it can’t be standardized. As such, it will never support the following cases:

1. Allow health insurers to predict and thus price the cost of psychiatric care.

2. Serve as an “impartial” basis for insurers to approve / deny claims in accordance with their pricing model and profit goals.

3. Create a defensible, scientific-seeming rationale for allowing some population monitored access to schedule III controlled substances while denying it to the wider public.

4. Allow wealthy, educated and self-directed individuals to learn the responses needed to obtain access to medically beneficial controlled substances without resorting to the black market.

Cynics, marxists and progressives would likely enumerate several additional cases, but the pattern remains the same. Survey-based psychological tools correlate weakly with patient outcomes and diagnostic accuracy and strongly with the politics and economics of private health care.

Sometimes.

Reply on news.ycombinator.com