Strong evidence is common (2021)
markxu.com
markxu.com
It usually combines a reference to Bayes Theorem, a deep hubris, and a significantly misunderstood concept from mathematics or technology.
Here, for example, it is clear the author does not grok EMH which applies across the population of trades not traders.
Usually this is done to provide the author or community with some claim to an inherent and unearned extraordinary nature, which justifies greater personal power. (You only need 200:1 evidence you’re extraordinary being the takeaway here.)
It is unclear why certain SV cliques continue to grant these people power.
Their intellectual insularity means that whatever value they once added in bringing logical rigor to conversations is now far outweighed by the train of collective bad logic powering their groupthink.
It is increasingly dangerous to have such people steer important societal debates.
These debates include crypto regulation where EAs famously associated with SBF, to “AI safety”, a field that appears to be invented out of whole cloth to allow LessWrongers to LARP as ML ethics and fairness experts.
For the general population though, you have the Freakanomics effect.
1. The intellectual mind's biggest motivator is certainty. This isn't necessarily a bad thing when certainty aligns with objective reality whatever that means, but then these folks get quite certain about how uncertain they are.
I would say at least 1 in 5 posts that get more then 100 karma are solid enough to stand up to ~1 hour of careful scrutiny, even by a reasonably well informed amateur.
It's the EA forum where things are a bit looser.
Some communities have positive signal. Others have negative signal.
The portion regarding the norms in EA communities argumentation habits is the most weighty part of the analysis. The EMH example is a supporting detail.
The thesis is this: EA communities are prone to insularity, hubris, and fabulism covered with a paper thin veneer of logic. The insularity generally means their arguments have not been subjected to even trivial scrutiny from non-EA communities. As a result, their arguments should as a first approximation be given no more weight than the average Reddit thread.
To be logically complete it would need to collect a variety of examples of hubris, fabulism, and insularity from EA arguments, which is left as an exercise to the reader.
Rhetorical fallacies are important to understand if you’re going to call them out. “Ad hominem” means attacking the argument by attacking the person.
Here I am attacking the general credibility of a community of people based on the example of one of their members’ arguments.
This is common in communities that are devoted to radical honesty and radical transparency, particularly those that actually have to hold these norms in high stakes market scenarios. It is called “believability weighting.”
For example, if the Norway Government Pension Fund historically provides poor logical arguments in their analysis of EU recessions, their arguments about EU recessions should be given less weight in the future.
Believability weighting is not the same as ad hominem.
By the late 1970s he was starting to regret it because he was now responsible for these people (one reason why slavery quit being economically viable.) Thus began David Miscavige's plan to bleed a few "whale" donors for all they are worth
https://tonyortega.org/2021/12/23/insider-how-scientologys-i...
Inequality has increased, you can't get blood from a stone, there is no point in a mass movement, so the goal has got to be target a few trust fund kids and get a whole lot out of them.
The Symbionese Liberation Army
https://en.wikipedia.org/wiki/Symbionese_Liberation_Army
almost got it right by kidnapping Patty Hearst except they chose a method of mind control that gets you immediately in trouble with the state. Today you would camp out at Oxford or Stanford and develop a small movement not because you really want a movement but you need a movement for the whales to be a part of. In fact you'd develop a network of movements so you can market test ideas and see what works most effectively.
It is so ironic that this group first filters people for a lack of critical thinking skills, and then puts them through a crash course in rhetoric so they can quickly say things like "Ad Hominem!" and then stick like a pitbull. The best way to deal with them online is to snipe at them and not get caught in the tarpitting they'll try to trap you in.
If you have good arguments to attack specific statements, methods, no need to attack people. Otherwise, it reminds me how tabacco / oil industry tries to discredit science (often successfully) by attacking individual scientists.
I see the effort to discredit EA lately (and given that I read about EA almost exclusively on HN, It says something). I wonder why.
See, this is an example of a contentless ad hominem.
Believability weighting is when you say, “Here’s proof this argument is bad. The community from which this argument originates makes these types of bad arguments frequently. Therefore we should believe that community less often.”
The second half of this article is complete nonsense because on one hand, he's seeming to arguing that extreme evidence is "easy to get", but it obviously can't be that easy when it's in a competitive environment where the number of people who could get that info is absolutely limited to something like only 1 percent of people.
Taking this quote:
> One implication of the Efficient Market Hypothesis (EMH) is that is it difficult to make money on the stock market. Generously, maybe only the top 1% of traders will be profitable. How difficult is it to get into the top 1% of traders? To be 50% sure you’re in the top 1%, you only need 200:1 evidence. This seemingly large odds ratio might be easy to get.
As other commenters pointed out, that is absolutely not what the EMH says - it doesn't say it's difficult to be profitable (it actually says the exact opposite, especially when taken together with modern portfolio theory), it says it's difficult (or, rather, impossible) to outperform the market.
But the more important point is that the author is implying that it's easy to get enough evidence to put you in the top 1%, as opposed to getting enough evidence to conclude that someone is in the top 1%, or to conclude that they're in any particular percentage actually. In other words, if you take his argument at face value, it may be possible to gather evidence to determine who is in the top 1% of traders, but that's very different from getting enough data to put yourself in that top 1%.
Another read is that it doesn’t take much to exploit other people.
Which tend to result in hubris.
> P(excess returns | skilled)
This is not at all clear. In fact, unusually high returns may simply be a result of a string of lucky, possibly correlated, gambles.
It may very well be that what separates a skilled investor has just as much or more to do with the ability to restrict the downside. Meaning that even when they lose money, they consistently lose less than the amateurs.
All too often I see authoritative figures dismiss presented evidence because their theories suggest it is (close to) impossible. It often goes like this:
A: "Hi my name is Mark Xu"
B: "Our models say it's practically impossible that this is true, so it must be an unfortunate glitch, or you are a liar. You don't have a PhD, so I guess I'll assume the latter"
Later...
"Despite rumors that the person Mark Xu has been spotted near South Park, Colorado, scientists say there is no evidence to suggest he really exists. Xu continues to be a mystery confounding scientists till this day...."
In the first case, prior probabilities are more often than not weakly informative. In the latter case, experts usually rely on a huge volume of information, beyond what most laypeople could even imagine.
In fact, we know as a historical fact that our current scientific theories are likely to be wrong or inaccurate. Your priors might be different from mine, but positing that odds of Bigfoot (or whatever) is less than one in a trillion or quadrillion isn't that useful.
Also, I hate to quibble about details, but someone saying their name is "Mark Xu" isn't really strong evidence that their driver's license says "Mark Xu". There are a lot of people who go by names other than their official name. In my case, I tell everyone my name is Ken, but it's actually Kenneth. In the case of Xu, many people from China use a Western name like Mark that isn't their real name. Edit: I just realized that the odds that someone's name matches their drivers license is at most 6:1, since only 84% of the driving-age US population has a drivers license at all.
The paragraph about the Efficient Market Hypothesis is just wrong. Contrary to the article, it is easy to make money on the stock market: historically it has gone up an average of 10% a year. It is also easy to beat the market average: you have roughly a 50-50 chance. How difficult is it to get into the top 1% of traders? Uh, that would be exactly 1 in 100. And I really don't see any connection between this paragraph and the topic of the post.
The post seems to have something to do with calibration training, but that link takes me to a signup page for some mystery site, which seems suspicious.
I'm trying to interpret this post generously and understand its message, but it seems kind of random.
Lots of words to say something either trivial or wrong, or both.
Bayes must be mentioned, because it's their religion.
And behold! Just clicking on the Bayes link takes me to another article that immediately mentions Yudkowsky.
I wonder what attracts internet people to Yudkowsky's brand of crackpottery, and specifically, what about Bayes elicits their quasi-religious awe.
All non main-characters always end up agreeing with the all knowing MC even though there are obvious gaps in the reasoning.
The point is to drive away anybody who has faculties of critical thinking. It's like those ads you see on the internet where they tell you that "TINNITUS IS DESTROYING YOUR BRAIN" or tell you what intermittent fasting plan is right for your shoe size and then send you to some two and a half long video that is there to weed out anybody who isn't incredibly credulous and after all that you find out it is some lame-ass supplement. Or it is like those poorly spelled letters that tell you the widow of some African dictator wants to to send you $15 million by Western Union.
Someone who's gotten through all that really is someone vulnerable who can be wrapped around someone's finger. People who are good at critical thinking aren't looking for the secret to become good critical thinkers.
Isn't it simpler to explain the situation by assuming he just got better at writing over time, and that stories teach lessons better than essays? It doesn't require a conspiracy to explain what you see.
But to each their own. They are doing what they enjoy I suppose.
I think it is because a lot of people think the human brain operates on Bayesian logic.
Thus maybe the 'quasi-religious', because it is seen as a clue to how we are conscious. The human 'spark' is really math, and it is Bayes. So when talking about human consciousness, it can start to sound maybe a little 'religious' in terminology.
I'm reading Anil Seth at moment, and it can be a compelling argument.
One problem is that practical reasoning is fundamentally probabilistic and nobody ever found a way to graft this onto logic programming or production rules that was completely satisfactory. For instance, a medical diagnosis system is trying to estimate the probability distribution of a hidden variable (the diagnosis) which is contingent on other hidden variables which are influenced by the signs and symptoms that you have observed and will be influenced by additional observations that you attempt to make, test orders, etc.
Bayesian networks, despite having some problems, are the most realistic approach to that problem and they really do work if the causal structure is fixed and you know what it is.
Thus I got interested in Bayesian statistics not because they are a model for human or animal thinking but because they are a model for things that can do some of the tasks humans do (particularly those that "expert systems" can do and in turning natural language into facts)
You could think of a task like "is this a picture of a dog?" as being a problem of sampling a probability distribution over a 3,000,000 dimensional space if you're doing it for a 1,000x1,000 pixel image. Done that naive way the problem looks completely impossible but the triumph of machine learning, particularly deep learning, is it can make a guess at that probability distribution given a fraction of the data it would take to sample it directly.
However, it appears that we do tend to round some priors to either 1 or 0. And it's specifically when we do that that we start to behave religously.
The next time somebody claims that the human brain operates according to bayesian logic, simply ask them what prior probability they (currently, until the next update) would assign to that claim.
A = a person is able to (correctly or not) quantify their priors
B = brain operates according to Bayesian logic
P( A ) ≈ P( A | B )
P( A ) ≈ P( A | B ) ≈ P( A | ¬B )
?
I suppose my argument is that if someone is using priors that are either a hard 1 or a hard 0, they've removed themselves from the ability to use Bayesian logic at all in any situation where data points in the opposite direction of their priors.
While you can still call it "Bayesian" if you insert a prior of 0, I think such an argument is a direct contradiction of the purpose of using Bayesian logic.
In other words, I would argue that P( ¬A | B ) ≈ 0. Refusual to admit a prior greater than a hard 0 is not compatible with Bayesian logic. Prior probabilities of exactly 0 should be seens as outside of the valid domain within Bayesian logic under most circustances.
The people theorizing that the brain is Bayesian are not saying that humans do it consciously. Like they can examine priors and making decisions.
It is just that the neurons in the brain update in a way that can be modeled roughly as Bayesian. It happens without us 'deciding to do it', it is just how the brain updates to process the environment. It is happening continually, as we take in senses, and update our internal model.
But if you start out with randomized priors, you will never reach 1 or 0 regardless of how much data you expose a bayesian system to. Humans, on the other hand, tend to fall into treating a probability as either 0 or 1 quite rapidly. For this step in particular, there seems to be something like rounding or L1 regularization going on.
Then, once they're stuck in 0 or 1 priors, people often revert to using evaluation similar to Bayes' Theorem again, but in that case the priors can no longer be updated (except through something like a psychological shock, hallucinogen etc).
But, as stated above, you cannot really reach such priors using Bayes' Theorem alone (if we assume the priors are not provided by genes or something that happens before the learning).
It seems humans do have miss-identifying visions. Where they categorize something fast but incorrectly, and then might need 'help/time/shock', to kick start re-categorizing. To re-see it again.
Or Like when seeing something for the very first time and they are 'befuddled', can't grasp it. Maybe like Bayes is having to iterate on it much longer.
I'm curious since I'm not all that familiar with Bayes. Is this what you are talking about with 1 or 0. People do make very rapid judgments, then settle on an a 'view'. Then it can take something to make them re-adjust.
It mostly has to do with the zeroes. Take Bayes' theorem:
P(A|B)=P(B|A)P(A)/P(B)
If you start with no knowledge, P(A) can be something like 1/n, where n is approximately the number of competing hypothesis.
B is the "data" part, a weighed compbination of all data used to update P(A).
P(A|B) is the posterior probability for A, given the data B. This posterior is then used to update your prior P(A) before you are exposed to new data.
Take the hypothesis "God exists as a male person". Before any data, maybe you give this a 1/4 prior probability. Add the Bible as your only source of data, P(A|B(ible)) may go to 9/10. Add all other religious texts, and maybe it goes to 1/2. Add all of Science, and maybe it goes a bit lower.
Now what happens when we set a probability to 0?
If P(A)=0, then P(A|B) stays at 0 regardless of how much data you pile into B. A prior of 0 makes it completely impossible to convince you that A is true.
Or, lets say you have two competing hypotheses:
A1: God is exists as a male person:
A2: Hypothesis A1 is false. (God either does not exist or is not male or not a person).
In other words, if you set P(A1) to 1 you simultaneously set P(A2) to 0.
And as above, when you have a prior of 0, it will never get updated using bayesian logic, so regardless of any evidence to the contrary, P(A1) will remain 1.
Only in a situation where P(B|A1) is also 0 could you ever doubt A1. When this happens, a person may live through severe cognitive dissonance or crisis, as if all of reality falls apart. The person will have to choose to discard A1, or to find some reason to ignore the data B.
For instance, if you believe 100% that some organization (any organization, but let's pick Hamas this time) are the good guys (hypothesis A), and some event B (like a massacre of civilians) seems to strongly oppose that belief.
Lets say P(B|A) is very low in this case.
If you have built your existence and identity around P(A)=1.
Now, instead of believing B really happened, you can introduce an alternative (conspiracy theory) view on the data, for instance:
C: It was really Mossad that tricked Hamas into attacking Israel.
Now, if you extend this, you can even turn this around. If you see A as the DATA not as the Hypothesis, you can set up
P(B|A) = P(A|B)P(B)/P(A)
P(C|A) = P(A|C)P(C)/P(A)
And this can cascade. If P(A|B) is small enough, then you can end up being certain that P(B) = 0 and eventually that P(C)=1.
Anyway, the above type of reasoning is at the root of religious and dogmatic thinking. In fact even a single false belief held with a prior probability of 1 can be used (through Bayesian logic) to prove almost anything.
And this is not limited to happening by accident. A skilled demagogue who can trick the audience into accepting (with no room for doubt) a single false or inaccurate claim, can then use to manipulate them into believing almost anything.
(Obviously, in the Hamas/Isreal conflict, this goes both ways.)
Humans aren't strictly math computations that can arrive at a absolute 0 or 1.
Maybe humans have a bit of a random number generator that kicks in some noise to avoid reaching absolute 0 or 1.
Allows some re-evaluation.
Maybe that would be option in AI to avoid the problem. During one of the games with AlphaGo it seemed to get stuck and was treading water making really 'plain or lackluster' moves. Until it kind of got unstuck. It seems to be similar?
Yes
> Humans aren't strictly math computations that can arrive at a absolute 0 or 1.
Actually, I think it's the other way around. Humans will arrive at a 0 or 1 (being convinced they know the truth), often with quite little support by evidence/data.
Rather, I think this is biology's "engineering solution" to having limited compute power.
> Allows some re-evaluation.
When you do get P=0 or P=1, you stop re-evaluating (when using basically Bayesian logic).
> Maybe that would be option in AI to avoid the problem.
AI can definitely be made to avoid this problem. It's a lot better at doing webs of Bayesian logic than humans are, if we tell it to, and can probably do that even if not explicitly designed for Bayesian logic.
My concern is more about humans.
The common word for people that evaluate P=0 or 1 AND draw the consequences from that, is fundamentalist. If anything, more intelligent people have a higher risk of becoming a fundementalist than others, since a single wrong absolute belief tends to cascade into a larger web of induced false beliefs, simply to keep their view on reality consistent.
Often it's better when people simply allow themselves to hold mutually exclusive beliefs, without caring so much about the contradiction. Like when people hold religious beliefs (maybe that people that don't belive in their God the way they do will go to hell), while NOT trying convert their souls at gunpoint out of "mercy" to prevent eternal suffering.
Another safe bet on the internet: unnecessarily caustic comments.
I don't object to people being passionate about niche hobbies -- I don't mock otakus or cosplayers for example -- I object to people LARPing as researchers and philosophers on topics they haven't made any serious contribution to, and particularly if they brand themselves as "rationalists" and on the path of being "less wrong" than others through their enlightened thought processes.
In other words, I mock crackpots and cultists.
As soon as you present this bet, my confidence that you are actually called Mark Xu goes down so low that I would not accept it.
Your confidence only dropped because of an intuitive recognition that the author presenting the bet is a signal the author previously lied about his name, as both the source of truth, the claim and motive to profit all lie with the author.
None of this is really relevant to the article, so I think it is safer to assume the author meant that some third party presented the bet. Following that, the author's logic makes sense.
> the author presenting the bet is a signal the author previously lied about his name
More generally: anyone presenting the bet is a signal that that person believes the author lied about his name.
“Extraordinary claims require extraordinary evidence, but extraordinary evidence might be more common than you think.”
I saw no evidence of that here. “Strong” is a very different qualifier than “extraordinary.”
And the choice of name does seem important. If someone with no visible appearance of being Asian gives you a very Asian name (or vice versa), you might have a lot more doubt. And if the name is otherwise humorous (Biggus Dickus) or stereotypical (John Doe) this also should affect your estimate. Why would it be a plus that "the posterior then doesn’t really depend a lot on what name he says"?
The point was that 1/n over all possible names is far from a reasonable prior, and that the Bayesian update barely depends on the name itself (within reason), but on other information gained during the introduction.
However, there is a decided lack of extraordinary evidence for extraordinary claims. That’s why they strike us as extraordinary. We’ve never seen them before. We’ve never seen anything like them. In fact all prior evidence has been strong evidence for ordinary claims.
The correct question is what are the odds someone is telling the truth about their name given your expecation that they will tell you their true name when asked. The odds of the name being Mark Xu is irrelevant unless I have some shortlist of names I am comparing against, and then I would use the odds of the name being on the shortlist, not the global odds.
It's obvious that the author's question is wrong because I would only be slightly surprised if I discovered that someone lied about their name, which is inconsistent with me having strong evidence for their name.
That said, it is frustrating to see something you know is a slam dunk and the model thinks it is 0.97 or 0.7 and if you look at how the calibration works it is often obvious why. The one that has frustrated me most is trying to calibrate full text search systems and never being able to get a confidence much better than 0.70, at least without special-purpose answerers.
https://mitpressbookstore.mit.edu/book/9781635617207
which is based on the fact that the odds to win are more accurate than the odds to place and show. If you go to the track you will see sometimes that the show odds for a horse are almost as good as the win odds but you have three to win so that is a great bet.
Dr. Z’s system constructs a probability estimator for place and show and based on the win odds, then computes the return on investment for the various horses and uses Kelley to decide how much to bet to maximize your returns with a low risk of going broke.
Card counters in Blackjack should use Kelly but either one of: (i) winning consistently or (ii) varying your bets a lot risks getting thrown out and doing both will get you a visit from a pit boss who, if you are lucky, will suggest you play roulette or craps instead.
There we go folks! Since I have now declared strong evidence to not be common you can believe on a 2,000,000:1 probability that in fact this is true!