Illusory Truth Effect | Hacker News Reader

Illusory Truth Effect

en.wikipedia.org

110 pointsvezycash55 comments

Open articleView on HN

There's also https://digest.bps.org.uk/2019/06/26/higher-intelligence-and...

(via https://news.ycombinator.com/item?id=20284218, but no comments there)

Has this been affected by the on-going replication crisis?

https://en.m.wikipedia.org/wiki/Replication_crisis

I wonder about that too. But I've heard about the effect a lot so it must be true ;-)

Perhaps this can be rationalized from a Bayesian context. Unless your prior is that you're 100% sure of something (which no one is) and you don't completely ignore what other people say, your posterior will eventually shift to the opposite side.

I think the error made here could also be described in Bayesian context: your posterior shifts to the other side because you keep counting repeated instances of the same information as new evidence.

That would be an error if you know the repeated evidence has correlation 1. The studies try to make this clear to the participants, but it's plausible that after a week they forgot where they read some of the repeated facts and treat them instead as independent information. I'd be interested to see how the interval of repetition affects the results.

"Sometimes people don’t want to hear the truth because they don’t want their illusions destroyed" --Friedrich

'Sometimes' seems to be a major understatement.

I'm kind-of surprised this didn't get studied earlier. This was one of the pillars of Nazi propaganda - or indeed, come to think of it, presumably any propaganda. Relentlessly repeat your lies, and eventually people believe them.

I wonder how this can easily backfire. Eg if someone repeats something repeatedly you can learn to believe it, but if you ever prove it wrong you never trust them again.

Reminds me of the Microsoft Halloween documents.

Yeah, I've heard about it already, probably true. ;-)

Does it take a twisted mind to apply that pattern to https://xkcd.com/552/ ?

>Does it take a twisted mind to apply that pattern

Not necessarily. They say the best liar is one who doesn't know that he's lying. Take the state of news today, we have bloggers who parrot a source with no verification and often zero comprehension of what they are reposting.

RMS's issue a few weeks ago exemplified this effect with probably no single twisted mastermind. An under age girl was forced to ask a billionaire pedo for sex who turned her down was shortened to. Billionaire pedo forced an underage girl for sex. The shortened sensationalized version motivated by advert money then spread like wildfire.

See the link below for how word of mouth stories become distorted with transmission. https://en.wikipedia.org/wiki/Rumor#1947_study

If you like this area of study I recommend studying object relations and reification.

Object relations is all about how we strive for consistency in how we observe the world. It helps explain how people react and understand things.

Reificaction is also a really cool tool to bring in since when people go around repeating some thing.

Hm, like "gig economy". Like you're supposed to sit there and just accept this loaded concept, which some may believe implies accepting falsehoods, let me explain:

Gig Economy: As if, you're powerless to shape environment. No statute could be created to make employement / labor more fair and stable (let alone generous). You can't influence it, analyze it, criticize it, lobby, vote. Somehow there isn't enough wealth to give everyone a living wage. Don't even bother. Let's imply the systems to fix it don't already exist - even when they do - and it could be done by the end of the year, a few months?

Reification is amazing because it's all about injecting life into abstract concepts. It's where we create and give meaning to new words - it's what comes before illusory truth repetition.

And it's amazing how things shift. For instance, USA - one big thing that makes us famous is big, fat paychecks. But today if you read the news and hear people speak, it's as if there's an implicit acceptance it's okay to give corporations welfare and consistency by the way of our laws, but not reciprocate by giving it to the workers?

Then sometimes you read that having better employment conditions and wages are "socialist"? Lol? You could say: Nope, capitalism is all about huge pay checks. We're sharing this success we created as a private collective of workers. And you're very welcome!

(And by the way, you all pay taxes that shore up the system, you're welcome for that too, as you're strengthening the system and helping downtrodden people you don't even know, you amazing humanist you)

When someone says "Gig economy", go "Wait a second, who said it was okay to accept that, and why are you referring to gigs as employement?", then you can say, "I define employment as pensioned, salaried, position paid enough to comfortably support a family 4"

Perhaps one could repeat "living wage" about 3 times every time they here "gig economy". Why tolerate anything less than comfort, stability and dignity for us? As if we lack the moral collective conscience and intelligence to do it? Lol?

> If you like this area of study I recommend studying object relations and reification.

> Object relations is all about how we strive for consistency in how we observe the world. It helps explain how people react and understand things.

I am very interested in this sort of thing, are there any resources or names you can recommend looking into?

I have a friend who is a lawyer. He seems to believe that rhetoric and logic are the same thing. Links like this bring me bad memories.

Should've thrown a dictionary at him.

I gave up arguing with him. He’s my friend and I love him. We play video games instead. No problem.

A lawyer’s job is not to actually find out the truth, like a scientist. A lawyer’s conclusion will not be tested by a harsh reality with which you can experiment. A lawyer’s job is entirely to convince other people of things. If people are convinced, the job is done; there are no further repercussions if the truth turns out to differ.

Therefore, your lawyer friend likely has never thought about what logic is, or what a scientist does, and how physical reality with experiments is a harsh and unforgiving master, and therefore believes that everything is what he does – i.e. rhetoric.

I believe you’re right. I gave him a book on informal logic. He never even opened it.

There is also a distinction between logical and quantitative thinking. Some people operate mainly out of a "whether or not" mindset, and they are not inclined to think in terms of "how much", "to what degree", or "how likely". Scientists tend to be quantitative thinkers, although computer scientists are quite often logical thinkers.

Agree in principle, but I have an issue with your use of the terms "logical/quantitative". A pedantic nitpick to be sure, but the "logical" mindset you describe is one of the most prevalent logical fallacies, that of the "false dilemma" [0]. Maybe there are even better ways to describe it than as a logical fallacy, but calling it "logical thinking" leads down the wrong track IMO.

[0] https://en.wikipedia.org/wiki/False_dilemma

He did not claim they were exclusive.

Your comment is an example of the fallacy fallacy.

Maybe so but I struggle to understand how, I must admit. In case you would care to elaborate I would be interested.

But unless you will argue that "logical thinking" is the correct term in this case I will submit that you have delivered a faulty argument yourself (since you invalidate my whole comment and not just the relevant part). Not sure which logical fallacy that represents though ...

The "fallacy fallacy" is valid for itself, though:

> since the fallacy fallacy is itself a fallacy, it cannot be used to label an argument's conclusion as false without committing it in the process. "You have used the fallacy fallacy, therefore you are wrong"

https://rationalwiki.org/wiki/Fallacy_fallacy

Maybe "black and white thinking" is what's being described.

That's even worse.

I know GGP wrote about "computer scientists", but I've seen similar things spoken (even here) about software engineers - that they have "binary thinking", they "reason in ones and zeros", etc. Which infuriates me to no end, because it's hard for me to think of another domain except hard sciences that forces you to think quantitatively, in numbers and orders and probabilities. Excelling in software requires the opposite of "black and white thinking".

Don't worry, I'm with you.

I think the stereotype is that Computer Scientists are black and white thinkers, and that's true for some. But a senior software engineer should be skilled at thinking with nuance and considering uncertainties, biases, etc.

I think that's imprecise. The opposition here is not between two kinds of reasoning, but between reasoning and persuasion.

Furthermore, quantitative thinking is itself logic.

Your comment reminded me of how many people think some tools, like IQ testing, are completely worthless because they're imperfect. An IQ score is not the same as intelligence, so it must be 100% unrelated to intelligence, in other words. This is not what's observed, to put it mildly.

Perhaps computer scientists, but software engineering is almost entirely about trade offs, so a software engineer who can't talk in those terms won't succeed at more than basic levels.

There is just the book for him: https://en.wikipedia.org/wiki/The_Art_of_Being_Right

Thanks. But I don’t that’s fixable in my friends case. It’s a long story :P

While I find the book to be written in a sarcastic tone, some people take it at face value. Your friend might be among them and enjoy the read in a very different way.

Choice quote: "and if he emerges victorious from a contest, he owes it very often not so much to the correctness of his judgment in stating his proposition, as to the cunning and address with which he defended it. "

Thanks... even though you suggested it for my friend, I think I'll just read it myself.

He's extremely intelligent (which makes his irrational tendencies more annoying hahaha), but he simply doesn't read anything I give him. Oh, well :P

It's particularly effective when combined with "Big Lie".

https://en.wikipedia.org/wiki/Big_lie

I don't follow. The "Big Lie" is a conspiracy theory devised (or at least advocated) by Hitler in order to justify antisemitism. It's not a true phenomenon -- like the Illusory Truth Effect -- but something Hitler falsely asserted the Jews did.

The “Big Lie” was the lie itself — it could not be publicly disproven under the Nazi propaganda machine, and therefore provides a cover story. Nobody really believed the cover story, but they were also too afraid to find out the truth because they knew it would get buried (along with them and probably their family). Such is the power of the “big lie”.

Yes, that's what I was trying to say. The "Big Lie" is not a cognitive bias, it's itself a lie. It's a myth Hitler either invented or popularized. So I don't follow how it relates to the topic under discussion.

I think it’s because Goebbels said something along the lines of “a lie repeated often enough becomes the truth” in reference to the Big Lie.

The cognitive bias is the effect of the big lie. Anti-semitism was already prevalent throughout Western culture at that time, and so the lie was easier to believe because of pre-existing cultural prejudice, and later because the propaganda was simply inescapable.

I'm truly not trying to be dense, but the "Big Lie" (according to Hitler) is something allegedly perpetrated by the Jews against Germany's military, by "blaming" it for the downfall of the nation. "The lie" to what you seem to be referring to doesn't seem to be the the "Big Lie" itself but simply antisemitism ("the Jews are to blame"). These are separate things. It requires antisemitism to believe the Big Lie exists and is perpetrated by "the Jews".

From Wikipedia:

> "Hitler believed the technique [i.e. the Big Lie] was used by Jews to blame Germany's loss in World War I on German general Erich Ludendorff, who was a prominent nationalist and antisemitic political leader in the Weimar Republic."

Maybe we're in violent agreement and I simply misunderstood what you're saying :)

edit: to clarify what I'm saying, there are two uses of "Big Lie" at play:

1- The narrative of "The Big Lie", the alleged lie "denounced" by Hitler. This was of course not a real thing, and Hitler made it up while looking for scapegoats and using antisemitism to blame the Jews.

2- The creation of the "Big Lie" itself by Hitler. This is a real thing, but we have a simple word for it: an antisemitic lie.

Judging from your other comments here, we may simply be using the same term in different ways - yours being the historically correct one, mine the colloquial one.

I don't think this confusion is unintentional. It's a classic fascist tactic to muddy the definition of words, phrases and symbols used by their opponents in order to force the conversation into emotional arguments. You can't have a rational argument when nobody agrees on the definition of the thing you're arguing about.

Today's "Big Lie" is simply less overt than that spread by the Nazis, and most people simply don't hear the dog-whistle of anti-semitism behind phrases like "New York liberals" or "cultural Marxism." But the modern day belief in a conspiracy of "globalist, left-wing elites" controlling society, academia and the media is just another, softer iteration on "the Jews" doing the same.

From the wiki reference: >"A big lie (German: große Lüge) is a propaganda technique and logical trick (fallacy)."

It is not a specific big lie, but a class of lies that exploit a certain mass naïveté.

> "It is not a specific big lie, but a class of lies that exploit a certain mass naïveté."

At face value, yes. In practice the term is exclusively tied to Hitler's meaning, who coined the term. I'm not even sure if "Big Lie" has ever been used in another context, possibly because the term is a Nazi invention. To be clear, the "mass naïveté" of "The Big Lie" is (alleged by Hitler to be) the populace's, who "believed the Jews" when they claimed (always according to Hitler) that Germany's loss in WWI was to be blamed on Ludendorff. There is no other real-world usage of the term. And we know this Big Lie was itself a lie coined by Hitler, and this alleged conspiracy to exploit the mass naïveté of the people didn't actually exist.

From Wikipedia:

> "The expression was coined by Adolf Hitler, when he dictated his 1925 book Mein Kampf, about the use of a lie so "colossal" that no one would believe that someone "could have the impudence to distort the truth so infamously". Hitler believed the technique was used by Jews to blame Germany's loss in World War I on German general Erich Ludendorff, who was a prominent nationalist and antisemitic political leader in the Weimar Republic."

and

> "Jeffrey Herf maintains that Goebbels and the Nazis used the big lie to turn long-standing anti-semitism into mass murder. Herf argues that the big lie was a narrative of an innocent, besieged Germany striking back at an "international Jewry", which it said started World War I."

So Big Lie, as a term, is irrevocably linked to the Nazis as its main meaning, is not a "general fallacy", and in fact it could be argued it doesn't describe a fallacy at all, but is itself a fallacy devised by Hitler.

When references disagree with you about a definition, it's time to realize that there can be more than one.

Can you point me to said references? I'm ready to acknowledge them, but I don't see them. The article in Wikipedia (linked to at the start of this thread) is exclusively about the use of the term by the Nazis and how they coined it.

The wikipedia article identified it as the name of a propaganda technique. There's nothing that restricts the use of that technique to anyone in particular. Consider that people other than the inventor of the hammer can use hammers.

Yes, I understand this, and that's something that sometimes happens with words. But the use of "Big Lie" in this context (i.e. capitalized, a concept about lying) is like the use of "Final Solution": sure, it could be used in another context different to what the Nazis meant by it, but how probable is that? All the examples in Wikipedia are Nazi-related.

I'll be happy to stand corrected if you can point me to a single use of this concept of "Big Lie" -- referred to unequivocally as the concept we're debating here, of telling a lie so big people won't accept it's not true -- that does not refer to the alleged "betrayal of Germany" meant by Hitler, or its related Nazi myths and propaganda.

At this point I'm only asking you for an example. To be honest I don't think there is, just as there is no other use of Final Solution, but of course I can be proven wrong.

That isn't hard at all. This one refers to it as technique used by the Soviet also.

- https://www.washingtonpost.com/news/made-by-history/wp/2018/...

Thanks for the example. I wasn't aware of the use of the term in US politics/journalism. In my defense, I'm not from the US and while I of course follow some of the drama they have going over there, I'm not always familiar with the details.

I'm glad I could help.

Shit. I got this all the time with my ex. Combine that with gaslighting[1] and you have a recipe for corn-fed, all-American propaganda.

If people repeat stuff back to you, make sure you repeat stuff back to them! Force feedback! Conglomerate!

[1] https://en.wikipedia.org/wiki/Gaslighting

Reply on news.ycombinator.com