Hacker News Reader

Comment by Bartweiss | Hacker News Reader

My primary concern with even cursory usage of Myers-Briggs is that retest consistency isn't as high as I'd like. Plenty of people retest very solidly, of course, but there's a large-percentage chunk of the population that walks or more boundaries and doesn't put up consistent tests. That makes it hard to say anything about the results, at least in simple "my type" terms.

A nuanced discussion would probably accept that these are continuous axes, but instead they get treated as clusters, distorting even casual analysis.

View on HN

Which is the fundamental failure. Picking E v. I says little about relative magnitude. 'E' might be slightly larger than 'I' but below the test's noise floor so you show up 'I' slightly more frequently. Or 'E' might significantly higher than 'I' so it's a consistent result.

Further, it says nothing about the fluctuations in magnitude over time, some days 'E' might be more important than 'I' and it really can swap on other days. Or even how accurately people self report.

Actually, a type (e.g. INFP) does say a lot about the magnitude of different cognitive processes. Take a look at this website (INFP linked as an example): http://www.cognitiveprocesses.com/16Types/INFP.cfm

There are only 4 bits of information. E v I, S v N, T v F, J v P.

So, if you are 'strongly' I and weakly P that does not change the order to show P is more dominate over J than I is over E. AKA, magnitude is literally not part of the test.

Notably, all the good secondary analysis of MBTI I've seen worries about magnitude. The RibbonFarm bit on it focuses almost exclusively on magnitude, discussing 'primary' traits rather than four-letter type clusters. A lot of the worst analysis, by contrast, insists on using non-magnitude, 16-case typing.

Which, of course, means that we're abandoning the actual test to get a substantive measurement.

When I took the MBTI as part of a college psych course, our test results came back with magnitude scores: moderately I, very N, barely T (ie, almost X), and very P. (The actual scores were numeric, but I only remember the relative strength years later.)

When discussing the results as a class, the strength of a type seemed to correlate reasonably with behavior, and explain more than the clusters on their own, eg, all the strong NPs had similar epistemological views and styles, fading towards an SJ style as the scores people had changed.

What a lot of people seem to forget, though, is the MBTI doesn't account for effects from disorders only healthy variation and it's about who you are right now, not who you are permanently. People vary over time.

A problem is that the opposite sides of each axis aren't even mutually exclusive. You can be a strong judger and perceiver. Or weak at both.

And the MBTI doesn't imply or assume axes. A type (e.g. INFP) is a label for your cognitive processes.

e.g. INFP leads with a primary Introverted Feeling (judging), an auxiliary Extroverted Intuition (perceiving), then Introverted Sensing, and finally Extroverted Feeling.

Some of the better tests will help identify these cognitive processes and then attempt to find the best fitting type based on them. You can read more about the cognitive functions here: http://www.cognitiveprocesses.com/Cognitive-Functions/index....

It's too bad that so many people see the types as diametrically opposed (e.g. Feeling vs Thinking -- you do both!)

I've retested several times, and while I'm normally I (strong), my other values were weaker, and fluctuated some (sometimes F, sometimes T, for example).

Amazingly, I've had arguments about this with folks. (probably E folks). They claimed that "it never changes", which is just patently untrue. Perhaps E folks don't introspect enough to re-evaluate their views on things? I know I tested different 5 years ago compared to when I took it 20 years ago.

I don't really see how you couldn't retest at least somewhat differently.

I've flipped at least two letters over time, and at least one letter on next-morning retest. J/P in particular has never seemed clear or significant to me.

I think this is actually the most common experience. The data I've seen says that scores on all four 'axes' are normally distributed. So it's actually fairly exceptional to have strong, predictable positions for every letter; cutting a bunch of normal distributions down the middle means most of us are on a boundary.

And yes, it's probably those silly E folks saying they never change. God forbid we wake up tomorrow testing as E, we'll have a paradox on our hands.

The folks who have argued that point with me over the years all happened to be strong E. I'm talking perhaps 3-4 people over 20 years - not a huge number to go on, but it was the only consistent I saw (male/female/old/young/etc).

Me: "Wow, I took this a few years ago and was a moderate F, but now I'm a slight T".

Them: "That's just not possible. It doesn't work that way. You just don't remember what you had before. It doesn't change".

Me: "WTF??"

Without the clusters, the tests would generate no interest. Imagine if most of us got the same results: "You are within a standard deviation of average. Go ahead and keep doing what you're doing." It would be like the typical middle-of-the-road performance review. We'd file it in the drawer and forget about it. The binary extremes provide the entertainment value needed to perpetuate the product, and also to justify the traits attached to the symbols.

> A nuanced discussion would probably accept that these are continuous axes, but instead they get treated as clusters, distorting even casual analysis.

As far as I understood the test, it results in your brain's _dominant_ mode on four axes. So you are either predominantly introverted or extraverted. There is nothing in the theory stating that you are introverted 100%, all the time.

I don't think I made my point clearly.

I'm not suggesting that MBTI claims we're 100% E or I. I'm saying that it assigns binary distributions on those axes, which would make it meaningful to talk about "E dominant" and "I dominant" people. Every study I've seen of the actual data, though, says that these are normally distributed traits.

So there are some people with distinctive, meaningful dominant modes on these axes. But the bulk of the population is at the center of the distributions, where marginal differences are being talked about like high-significance bimodal results.

This isn't inevitably the case, but we live in a world where MBTI results are still being used to determine who's suited to which jobs. If you're hiring sales guys who are 51% extroverted and refusing ones who are 49% extroverted, that supposed binary distribution is misleading you badly.

I thought the MBTI gave an X if you were actually at the middle?

So you'd get an I at less than 45%, an E at more than 55%, and an X if you were on the border.

Further, you said they were normally distributed, but didnt talk about the variance. It's possible that people who are just a little from the middle by % of population are meaningfully different than the average -- a quarter standard deviation could be a meaningful amount of change in the real world.

So your argument doesn't actually support your claim: with the inclusion of an X band, it's possible the gap between E and I is meaningful in real world situations. We can't know merely by knowing it's normally distributed, we need to analyze the standard deviation size versus the minimal meangingful difference in the real world.

I'm oversimplifying, yes.

Several instances I've seen of MBTI didn't offer X, but that's definitely an improvement. The one I got for high school career services actually crashed if you 'tied', because that hadn't been accounted for at all.

Without the X band, "normal distribution" is enough to say things are broken. It means that you're consistently dividing similar people while grouping dissimilar people, which isn't what I like in a cluster.

With the X band, SD becomes important and a wide distribution might provide good results. Anecdotally, this could match the high attention given to E/I versus F/T - high SD axes would be relevant to more people.

Having said that, I'm still skeptical of much of the MBTI literature. It definitely deals in bimodal distributions, asserting that these clusterings are features of the territory rather than the map. If the peak frequency is a value the test can't inform about, I think that still invalidates much of the literature even if the test has some predictive power.

> Several instances I've seen of MBTI didn't offer X, but that's definitely an improvement. The one I got for high school career services actually crashed if you 'tied', because that hadn't been accounted for at all.

While I get the criticism of those tests as poorly implemented pop-psych, it only seems fair to attack the MBTI itself as a concept if you criticize the actual written, send-in-and-evaluate official version. The version I took of that (which came with a ton of propaganda from the testing company) featured both an X band and numeric strength scores for each category. It also featured a lot more questions than the typical online free ones, and had redundant questions the way that standard psych evals do. So at least some of the "professional" MBTI people are providing that kind of test, even if their analysis is all about bimodals and nonsense.

> With the X band, SD becomes important and a wide distribution might provide good results. Anecdotally, this could match the high attention given to E/I versus F/T - high SD axes would be relevant to more people.

My main point was that the joint distribution increases the spread away from XXXX to things like XXFX or XNXX, where even if you're still near the exact middle, there's less people exactly at average, and so the test is more likely to relay at least one fact in one category about the test taker than any particular category is to say something useful.

So even though the test can't say anything about the hyperbox in the middle of the range, it is relatively small because getting a little way away from the exact center likely says something about at least one category, even if the majority still can't have anything useful said about them.

Think of it this way: assume that the test can't say anything about the middle 20% of people for a given trait, and that the traits are independent (ie, how you score on one has nothing to do with the others). The odds that you're XXXX is (0.2)^4 = 0.0016. So only ~1 in 500 people would get nothing out the MBTI, even though ~1 in 5 has at least one category with an X.

And that's assuming a relatively large X band. Drop the X band to 5%, and it's only ~1 in 200,000 that would get XXXX.

> Having said that, I'm still skeptical of much of the MBTI literature.

I threw out the marketing booklet mostly unread after the statistical nonsense of the first few pages was too much when I paid to take the professional version of the test in school.

The test is alright, but not great and modern data analysis can probably do a much better PCA than the MBTI is, but the company is outright moronic in what they say. And others can be much worse.

The big problem of MBTI is the axes aren't well aligned with supportable dimensions of personality, though they have some correlation with them. Models like EPQ (3 factors), Big Five (5 factors), and HEXACO (6 factors) have better methodology behind them (though each still has strengths and weaknesses).

MBTI is a just-so story with a lot of marketing behind it.

The goal of the test is just to give an approximation. The test is not accurate (and is not claiming to be).

The core of the theory is to highlight natural preferences in the way we use cognitive functions. It's a pretty good and useful model. It's not a definite one though, like all the models we have it's meant to be improved and replaced over the years (like in physics).

99% of the critics about MBTI are focusing on the tests (which I agree are not accurate) and the authors don't understand the theory behind it - which is the really useful part. (Carl Jung's work, cognitive functions preferences, etc)

I think there's a cross-purposes argument happening here.

I, for one, am totally alright with MBTI-style questions as a tool for analysis/introspection. RibbonFarm has some great writing about identifying 'dominant' cognitive functions based on magnitude data from MBTI.

Strictly accurate? No. A useful framework for planning? Absolutely.

MBTI is also an organization, though. It charges money for training, issues education materials, and gives trainers strict guidelines on what assessments are ok to make. That system is hideously inaccurate and deserves criticism. It asserts that the test is accurate to reality, that the personality types are 'opposed', and that people are bimodal on the four axes. Many of those claims explicitly contradict the Jungian theories that inspired them.

So it's easy to overreach when criticizing the MB model, and most articles like this aren't clear about their target. But the actual MBTI organization - which is still popular with big companies in its original form - is wrong in terms of its own theoretical foundations, defended with bad data, and deserves extensive criticism.

Agreeing 100% with you.

Reply on news.ycombinator.com