Hacker News Reader

Comment by jamesrcole | Hacker News Reader

This post is about "the compulsion to replace quantities with dichotomies (‘black-and-white thinking’), even when such dichotomization is unnecessary and misleading for inference" -- I don't understand why they're calling this deterministic thinking!

As far as I can see the phenomenon they're talking about has nothing to do with the quality of being deterministic, and I didn't notice any explanation of why they've chosen that particular word. Is there some other notion of deterministic aside from the usual one?

View on HN

Once you understand that the author is talking about how people understand the world, it makes more sense. I actually see some parallels here to the problem of induction, i.e. can we predict the future by generalizing a finite number of past events. The answer is going to be "no", which most people would agree with if you engaged them in a dialog about the idea. Yet most people will act as if they can predict the future based on the past.

One example might be "I had a bad experience using $SOFTWARE, so if you use $SOFTWARE then our project is going to fail and it's going to suck". You're thinking "If X then Y", not "If X then P(Y) > n". I.e. you're thinking in a purely deterministic way, when the evidence really only supports a probability.

This way of thinking is ultimately due to a deterministic model of how the world works, rather than a probabilistic one.

I'm not sure if I follow your example.

Are you implying after one occurrence a person thinking "in a deterministic way" is going to take the outcome and think it will reoccur again?

I don't think that is deterministic thinking because a determinists accounts all the variables that made the outcome. Such as how the software was used and understands that will be different for the outcome of some other user.

Maybe the example could be a bit clearer. In the example, the point is that they don't know all of the variables. I would define this sort of (faulty) deterministic reasoning literally as not accounting for hidden variables. If we know absolutely every variable and potential confounding factor, then deterministic reasoning works and is valid. The problem is that in pretty much every case we don't know all of the variables, and so we can't generalize from a finite number of cases. The solution that the author is giving is to think in probabilities instead.

If you read to the end, there's a mention of the premature collapse of the probabilistic inference about some entity. That clarifies what the author is talking about.

The scenario is something like a situation in which individuals are categorized into two discrete groups, and then inference proceeds as if the fate of each individual has been determined by which group they fell into (rather than determined probabilistically). The fact that a discrete categorization is assumed to 'determine' the outcome (the possibility wave function 'collapses' at the moment of categorization) could certainly be referred to as deterministic. I agree that it's not an application of the word that we are familiar with.

Maybe 'fuzzy set' categorization would be better than 'deterministic'?

He mentions that “deterministic” means “discrete” or dichotomized, presumably opposed to probabilistic or “real” values.

I found it funny he spends the entire article eliciting a dichotomy between thinking “deterministically” (dichotomized) and thinking non-deterministically.. yet chiefly to form the suggestion that one should not make such dichotomies to better understand things.

How could we model the co-existence of the discrete and dichotomized with the continuous and probabilistic... without involving at least one dichotomy?

I found the term "deterministic" confusing too, probably because I'm more familiar with it being used in a different context.

He uses the phrase "inappropriate discretization", which fits better with what he's describing as a problem in how we think.

To me it seems to boil down to "duality" being a fundamental property/strategy of thinking. Buddhist philosophy points this out in numerous ways, and the paradox/contradiction inherent in it.

From the article's conclusion:

> ..When we’re talking about the problems of deterministic thinking, or premature collapse of the “wave function” of inferential uncertainty, we really are talking about a failure to incorporate enough of a continuous view of the world in our mental model.

I think the point is that we should be more willing to accept uncertainty and probabilities in thinking about the world, rather than build mental models on "false certainties" and "dichotomania" that reduces the world to what we think we know and understand.

I think the author say some strange things, I am not entirely sure he uses the words correctly but here is my strongest take on what he is trying to say:

Lets say you have an internal value function F which you use to judge peoples worth. If F is deterministic it means that for any observation X it produces a value F(X) = V, where V is some number.

With this function a rational person would be racist, as statistically whites are more educated than blacks, so maybe F(WHITE) = 0.7 and F(BLACK) = 0.4. People from this category say things like "Of course he failed the maths test, he is black!".

But lets say that your F output a probability distribution instead, then you'd recognize that the information is incomplete, so even though E(F(WHITE)) = 0.7 it could both be lower and higher. With this way to see the world the statement "Of course he failed the maths test, he is black!" doesn't make sense, as there are lots of black people who do very well in maths so your value function would include those possibilities as well.

Note that the talk about discrete categorization doesn't make sense, having real valued prejudice function based on loads of factors is still deterministic.

In my experience most people go with the deterministic value function when not pressured. But when you say something like "Do you really mean that no black person could have done well on that test?" they typically acknowledge that there are some black people who would have done well, so they can still think probabilistic but it takes more effort. Maybe said effort is too much for the average person to apply it everyday so we are doomed to experience the current political strife forever.

Yet we are not ruled by Genghis Khan or Hitler. How come?

I don't see your point. People still hate each other due to trivialities, we just mostly stopped killing each other.

If you wonder how we solved racism, we did it with simple and blunt rules everyone can follow such as "don¨t say the n-word" or "don't say that whites are better than blacks". People are still racist, but simple rules like that makes it harder to rile up a mob enough to burn down a black neighborhood (I know it happened in Tulsa, that is why I used it as an example since it probably wouldn't happen today).

Your reasoning was fine until you jumped to your "doomed forever" conclusion. Clearly nothing is doomed forever if things like killing and racism are reducing.

Your theory explains why people misunderstand each other and are "doomed" to states of conflict due to how their minds work. It needs to go further to explain how good outcomes are produced despite conflict and misunderstanding.

Why do they accept "simple and blunt rules"? Where are these rules coming from? Why aren't the rules pushing people in the opposite direction? Are there other mechanisms beyond imposing simple and blunt rules on everyone? etc etc etc Don't be satisfied with your theory. It's a good start but keep digging.

Probably the most common question in applied statistics is "Does X affect Y?" To answer this, you get data and do some analysis that, from the bayesian perspective, leads you to some distribution of beliefs over how X affects Y. From a non-bayesian perspective you still have some kind of uncertainty. But you still want to turn that into a yes/no, so you come up with a rule for turning that uncertainty into a deterministic yes/no. Data in, yes or no out.

What has it being yes/no got to do with being deterministic? What definition of 'deterministic' are you using here?

Yep the article title is NOT 100% clear for those who have not worked with stochastics before. They mean that while the world is stochastic (randomly distributed) we think of it deterministically. In their example of pharmaceutical trials, the FDA requires a "hard" (deterministic) answer, while even the most thorough results remain stochastic (i.e. some distribution).

What definition of "deterministic" are you using here?

Deterministic in the sense of determining things to their "value" instead of leaving them as a continuous probability.

Discrete is the word for that.

There is also determinant. Which might have been a better choice.

It's also deterministic in the sense that people who think this way are assuming that the universe works deterministically, so they can know that A always leads to B, etc

Reply on news.ycombinator.com