Hacker News Reader

Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately.

As for the rest, yes, I agree with you. Often, after having considered a great deal of evidence, I condense the finding into some strength of belief about something. Now, absent the original evidence I have a position. This is usually fine since I actually lack the ability to summon all evidence on demand.

It's okay, though, since I (and I suspect most people intuitively) have both a likelihood notion and a certainty in likelihood notion analogous to a Confidence Interval and P-value, if you will.

So I might hold the belief that GME is going to be between 120 and infinity on Jan 13 2022 and have a certainty in that belief that is like 10% but lose the evidence that made me think that.

This is also sort of a failure mode for fast Aumann Agreement even in co-incentivized individuals - the fact that memory is short.

So the problem is that my certainty holding is not well done.

Thanks for the thought prompt.

I've always found it odd that the people called "rationalists" are not rationalist. Talking about evidence and realistic priors all the time is good (they don't do enough evidence gathering though) but it is not rationalism, it is empiricism.

Surely a rationalist would be into logic and actively refuse to read studies, like the Austrian economists did.

It's just a terminology thing. Sort of like how atheist actuaries still account for Acts of God.

The word 'rational' is one of the most abused in western intellectual history. Hegel considered his system 'rational' and I think it is fairly self-evident that that has nothing to do with these internet 'rationalists'.

> Rationality as a tool to serve yourself. A lot of SSC and AC10 is around this idea that rational intelligent thinking with self awareness of bias allows you to model the world accurately.

That sounds like an extraordinary claim that would require extraordinary evidence. No doubt physics and engineering have made great advances, but that doesn't mean that every social science that dresses itself up in the language of rationalism will be able to make similar advances, or really any advance at all. In fact, there is a case it will lead to regression rather than advancement:

Societies are very complex things made of people who respond to changing incentives. I don't believe anyone understands them very well, nor has any field in the social sciences established a track record of making highly accurate out of sample predictions, especially for the efficacy of interventions 10, 100, 1000 generations on. Even the simple question of "what keeps a society together?" is still a puzzling one, especially now that post-modern Western societies are disintegrating before our eyes and are failing to even biologically reproduce at replacement levels.

And so we turn to values. At least with values, it is possible to gain experimental knowledge in the laboratory of history. Societies reproduce values and compete with each other as well as with a hostile environment, so long lived societies should have acquired value systems more conducive to social survival.

And I don't think we can make much stronger claims than that.

Certainly someone just imagining stuff or doing some small experiments is unlikely to come up with robust rules for organizing society that will stand the test of time better than what comes from the long lived traditions of successful societies.

Someone living 1000 or even 100 years ago did not have the information we have.(eta: to design a society) Technology also lets us afford things that past societies simply couldn't do.

We sometime learn from evolved biology to make machines, but sometimes we can also see that evolution made a bad design. By learning from what has worked it's not too difficult to design an alternate society that will work just fine for decades or centuries.

Some societies survive by preying on others. Some are stable but prey on their own. I don't want a life in service to a society that evolved to be strong. I want a society that exists for the benefit of the people in it, and improves as we learn better ways to do things. (The current system does change but different structures could be much more adaptable.)

I should clarify. I meant "more accurately than not engaging in that pattern of thought" not "accurately". The latter is an absolute statement, the former a relative one.

And, yes, one can use alternative schools of epistemology if one believes so. This group of people derive theirs significantly from Popper's line of "critical rationalism" which is different from "empiricism" (the names here are the ones that annoyed our cousin comment over there).

This school has certain affordances that permit them to reason as they do but if you disagree the differences between us are likely so large that it is unlikely that I can explain without expending far more effort than I am willing to.

Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values.

So, for example, in the climate change example he gives, while it is of course appropriate to take into account possible biases of people promoting one position or another, and ways that those may result from the different values that the people with those biases, one should nonetheless endeavor to have a tendency to believe what is actually true about the factual matter, even if it were that the people with the wrong values were biased as an (indirect) result of those values, in a way that led them to be more likely to reach the correct conclusion about the empirical facts, than those with the good values.

(Not to say that I think this is the case in this example. Just, hypothetically. Or, what do you call something that is like a hypothetical but without expressing any position about whether the thing is actually true?)

In addition, while they may get tangled up and which is which may change for a given person over time (due to human thought being fuzzy and wobbly), I think it still makes sense to make a distinction between terminal values and instrumental values. Further, if one values something instrumentally, because one believes that it is instrumentally beneficial for one's terminal values, and it is demonstrably true that it actually does not benefit one's terminal values, then one should come to believe this truth, and so no longer instrumentally value the thing.

I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model.

People will sometimes understand their past behavior as arising from a cognitive error, and endeavor to address this. I suppose you could regard this as simply conflicting preferences, or preferences about preferences, but I don't think this is the best way to understand things.

Even if you shouldn't be convinced of a terminal value thing based on evidence, doesn't mean you shouldn't be convinced of an empirical thing that happens to be associated with or championed by those who are promoting different values.

That's true, and I have been convinced of some of those things. For example, I am empirically convinced that mass shootings by spree killers are shocking but not a major issue of public health or safety in the US. Even though I believe that Democratic politics gives disproportionate attention to mass shootings, weirdly mirroring how Republican politics gives disproportionate attention to Islamist terrorism, I'm not going to say so in the company of many Democrats. Because another thing I have been empirically convinced of is that people build effective political coalitions by emphasizing common ground and judiciously overlooking their allies' faults. I want action on climate issues, so I'm not going to alienate my natural allies by lecturing them about their irrational fear of semi-automatic rifles. Effective politics is almost the thematic opposite of a personal quest to aggressively probe all things in search of Truth.

I also don't think cognitive error can be entirely reduced in all cases to revealed preferences, because, uh, -- well, ok, one can, of course, model all behavior as arising from a preference to take exactly the sequence of actions that one takes. But I think you will agree that this is not a reasonable model.

It's a model that doesn't lend itself to mathematically grounded predictions. But is it actually not reasonable? Without access to another person's qualia it's really easy to misattribute whether their more surprising decisions are revealing a preference, demonstrating cognitive error, or servicing a fundamentally different value orientation. Like Hume's Problem of Induction I can't effectively grapple with it but I can't exactly dismiss it either.

> It is probably a good strategy to refuse to be talked into positions against your fundamental values by mere evidence. I'm joking but also serious.

I believe that too. There are some boxes so dangerous we need people not to open them even if they believe they should.

However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence.

We don't get sewage, skyscrapers and subways by following our instincts or because some people somehow made them values. Everyone wants good sewage systems. The best parts of society come from following evidence and logic as best we are able - including, I point out, the legal system.

However there are a lot of things in the world that we can be wrong about. A strategy of "I'm going to use deontological reasoning" just means that we have to sort right from wrong by an evolutionary process over generations, getting to kids while they are young enough to be swayed, in a process that is a bit icky and disrespectful of people's beliefs. It would be better if people could be persuaded by evidence.

I don't think that you necessarily need to sway kids while they're young. 10 years ago there were no states in the US that had legalized recreational cannabis use for adults. Today there are 17, and I expect at least another 17 over the next decade. What changed? Attitudes shifted among all age groups. It didn't seem like a process of grandparents dying out and their grandchildren entering the voting booth.

It didn't seem like an evidence based process either. I was well persuaded in the 1990s that the deleterious health effects and other negative externalities of cannabis use were no worse than those corresponding to alcohol. The historical example of Prohibition was more than enough evidence for me that fueling a black market was worse than letting adults impair themselves. I wonder why evidence didn't persuade people to change laws sooner, yet the shift happened far faster than generational replacement once it started.

If anyone reading has a convincing model for how the cannabis attitude shift happened: does it only rationalize history, or does it also predict other things likely to change in the near future? There are a lot of ways we can turn past series of events into coherent stories. There are fewer that successfully foretell what happens in the next chapter.

The cannabis legalization happened because of two additional factors:

A) Overincarceration of non-violent offenders

B) the reveal of racist roots in the foundational war on drugs rhetoric (which led to more strict penalties for the use of crack [cheaper street drug], for instance, than straight cocaine [high-society, white-collar drug]).

At least that seems to be the lynchpin from my point of view. It was always going to fail from mmy point of view since prohibitions in the U.S. don't have stellar records.

> There are some boxes so dangerous we need people not to open them even if they believe they should.

Religion is several of those boxes, which have already been opened for centuries.

Faith, (by which I mean deliberately attempting to believe something to a degree of certainty which exceeds what is warranted by the available evidence) is one of those boxes that should be sealed up and ejected into space, or at least made to be deemed immoral by the majority of people, since it obviously is immoral.

Few people carry that definition of faith though, but many, especially during the French Revolution, felt as you do, and acted on it, not by ejecting religious people into space, but by gathering them together in barns and burning them in mass executions.

https://en.m.wikipedia.org/wiki/Infernal_columns

The cognitive error being discussed here relates to a systematic failures to make accurate descriptive statements. The distinction between deontological and consequentialist thinking is entirely prescriptive. Being able to make accurate descriptive statements and understanding the accuracy of descriptive statements are valuable tools regardless of if you are a consequentialist, a deontologist or adhere to some other moral framework altogether.

I believe that torture is wrong and I also believe that torture doesn't work. I could theoretically be convinced that torture works, but I would still believe that it is wrong. If I were to insist that torture doesn't work in the face of strong evidence to the contrary because of my moral beliefs that would be a cognitive error.

Regarding your footnote: They cannot be the same thing because preferences are prescriptive and errors are (at least as they are being discussed here) are descriptive.

> "cognitive error" and "revealed preference" may also be different names for the same thing. Are they experimentally distinguishable?

"I would still be against torture no matter what the research shows" is revealed preference, and "I'm against it, therefore it doesn't work, no matter what the research shows" is cognitive error.

This is clearer if you suppose that I published a good, thorough study showing that torture works very well. In the former case, you could accept it as accurate but retain your opinion; in the latter case you would be forced to either change your stance or accuse me of [ineptitude, fraud, bias, secret sadism, etc].

I think this is often confused because "works" is poorly defined leading both sides to interpret it as: works == achieves my aims == matches my values

I believe that the author makes the basic assumption that rationality is the end.

> But in fact many political zealots never accept reality. It's not just that they're inherently skeptical of what the other party says. It's that even when something is proven beyond a shadow of a doubt, they still won't believe it.

“We’re an empire now, and when we act, we create our own reality. And while you’re studying that reality — judiciously, as you will — we’ll act again, creating other new realities, which you can study too, and that’s how things will sort out. We’re history’s actors . . . and you, all of you, will be left to just study what we do.”

- Karl Rove