Hacker News Reader

Comment by comp_throw7 | Hacker News Reader

This is a deeply confused set of arguments.

There's no fundamental ontological difference between "give me your likelihood ratios between drawing cards [a and b] as the next draw from this deck of cards" and "give me your likelihood ratios between finding and not finding a continent in an unexplored part of the world". While what we ultimately care about is the territory, which is fixed, we only have our imperfect map of the world with which to model it. There is nothing inherently "probabilistic" about a shuffled deck of cards - the cards have a specific configuration; the uncertainy is only present in the observer(s). Predicting the next draw from a deck of cards is subject to "out of context" model violations the same way that any other prediction over future world states is.

The proposed escape hatch seems to be to dodge the question, but of course in reality you are not agnostic over all possible future events that haven't been ruled out by being "bad explanations". You can say, "well, don't make predictions with numbers on them", but by acting in the world you are making predictions with numbers on them all the time! Bayesian epistemology doesn't claim to be perfect, it claims to be the least inaccurate way of making those predictions and updating on new evidence. Now, there's a separate question of how bounded reasoners (i.e. humans) can best accomplish their goals, and "explicitly run Bayes to decide whether to get out of bed" probably isn't it. But superforecasters are an existence proof that explicit Bayesian calculations are still extremely useful even in highly uncertain domains.

Mechanistic explanations are great but even having a mechanistic explanation doesn't let you say P(x) = 1. There can be enormous differences in reasonable mitigations to take between P(x) = 0.99 and 0.999999, depending on context.

View on HN

> There's no fundamental ontological difference between "give me your likelihood ratios between drawing cards [a and b] as the next draw from this deck of cards" and "give me your likelihood ratios between finding and not finding a continent in an unexplored part of the world".

Can you explain this claim? It seems very clearly false to me, so perhaps we're thinking about things in very different terms here. To me, the extremely obvious and fundamental difference between the two is that we have a very good explanation (to use Deutsch's favorite term for emphasis, although that terminology isn't strictly important here) for why we know the probabilities of various events in a game of cards. Of course, our explanation of why we know those probabilities does depend on some assumptions of our model, namely some things about how the cards are shuffled and dealt, but the explanation doesn't really have anything to do with modeling subjective uncertainty of a particular observer. Sure, it is probably physically possible for some observer to have more certainty, like someone just outside the room with an extremely sensitive radar, but that observer would likely be understood to be violating the rules of the game (just like any card player who cheats using more feasible methods).

To me, that's fundamentally different from invoking probability when discussing things like the existence of a specific thing in a specific unexplored part of the world, again assuming we don't have some explanation for why the existence of that thing in that place actually has a probability associated with it.

I'd have a hard time doing better than Jaynes:

> Suppose you have a penny and you are allowed to examine it carefully, convince yourself that it's an honest coin; i.e. accurately round, with head and tail, and a center of gravity where it ought to be. Then, you're asked to assign a probability that this coin will come up heads on the first toss. I'm sure you'll say 1/2.

> Now, suppose you are asked to assign a probability to the proposition that there was once life on Mars. Well, I don't know what your opinion is there but on the basis of all the things that I have read on the subject, I would again say about 1/2 for the probability. But, even though I have assigned the same "external" probabilities to them, I have a very different "internal" state of knowledge about those propositions.

> To see this, imagine the effect of getting new information. Suppose we tossed the coin 5 times and it comes up tails every time. You ask me what's my probability for heads on the next throw; I'll still say 1/2. But if you tell me one more fact about Mars, I'm ready to change my probability assignment completely. There is something which makes my state of belief very stable in the case of the penny, but very unstable in the case of Mars.

Source: https://books.google.com/books?id=tTN4HuUNXjgC&pg=PA553&lpg=...

Ultimately it's a question of how much each bit of evidence would cause you to update (that is, difference of degree, rather than kind).

But both of those cases are concerning the “Bayesian epistemology” side of things, not using probability to reason about how to bet in games of chance with well-established rules and mechanics. The usage of probability Deutsch is approving of is the latter, and is unrelated to a particular observer’s level of confidence that the cards are designed fairly or that the dealer and other players are playing fairly.

> Ultimately it's a question of how much each bit of evidence would cause you to update (that is, difference of degree, rather than kind).

Isn't this only because you have strong priors for the coin and weak ones for Mars life? Suppose you didn't have your example's information about the coin. Then the situations are the same and a new piece of evidence in either case (e.g., discovering whether the coin is round, has two heads, or where its center of gravity is).

There is no such thing as a "strong" prior or a "weak" prior. A prior is jut a number between 0 and 1.

But the coin example is really not very apt because there is potential information that will change your belief about the coin. For example, you might discover that the coin is magnetic, and there is an external field being generated somewhere in just the right way to make the coin land heads every time. Or something like that. Whatever the case, if you get (say) 100 heads in a row you should start to have some serious doubts about your initial analysis of the situation.

But ultimately, it should take more surprising evidence to shake up your beliefs about the coin that it would to shake up your beliefs about Mars, right?

So, the prior which is strong/weak is the probability distribution of probability distributions. There is a <<1% chance that I will ever believe there is a >90% chance of flipping heads. But there is a much greater probability that I will one day believe there is a >90% chance of life on Mars.

What terminology would you use to describe that, if not strong/weak priors?

> it should take more surprising evidence to shake up your beliefs about the coin that it would to shake up your beliefs about Mars, right?

I would say it's the exact opposite. Encountering an unfair coin would not surprise me a bit because I know, for example, that double-headed coins exist. But encountering life on Mars would surprise me a lot because I've accumulated a lot of evidence already that there isn't any. (There might have been in the past, but there isn't any now.)

The only thing that I consider "surprising" is encountering evidence that moves one of my priors from being close to 0 to being close to 1 or vice versa. That happens less and less as time goes by because (I hypothesize) my beliefs converge to reality, and reality is stable in the sense that there is something out there towards which it is possible to converge. But at no stage in this process are my beliefs about propositions modeled by anything other than numbers between 0 and 1, with 0 meaning, "I am absolutely certain this proposition is false", 1 meaning, "I am absolutely certain this proposition is true", and 0.5 meaning, "I have absolutely no idea whether or not this proposition is true."

How does this work for the reasoning you just gave? Was there some evidence that moved your priors about Bayesian reasoning from near 0 to close to 1?

Apply that to any philosophical or mathematical reasoning. Let's take an interpretation of QM. How would you assign a probability to Many Worlds Interpretation? Do you have some way of assigning your degree of belief between 0 and 1? Or are there just arguments you find more or less convincing for it?

There was a thread on here the other day about black hole information paradox and how we may never be able to test any mathematical reasoning regarding what happens to the information inside a black hole. Can you assign a Bayesian probability to that?

> Was there some evidence that moved your priors about Bayesian reasoning from near 0 to close to 1?

I think you misinterpreted what I said. My belief in Bayesian reasoning was very nearly 1 from the first time I encountered Bayes's theorem. Nothing I've seen since has changed this, i.e. nothing about Bayes's theorem has surprised me.

The last time I can remember being surprised (in a Bayesian sense) about anything was when Donald Trump won the 2016 election. My prior on his winning was very close to zero when he announced his candidacy (and frankly I think his was too), and obviously my posterior after the election was very close to 1. (Contrast this with the reasoning currently being deployed by proponents of the Big Lie.) That event really rocked my world, but I can't think of anything that has surprised me in the same way since then.

> How would you assign a probability to Many Worlds Interpretation?

The same way you assign a probability to any proposition: you look at the evidence and decide whether the MWI is a good explanation that accounts for all the evidence. I did a very deep dive into this exact question about three years ago:

https://blog.rongarret.info/2019/07/the-trouble-with-many-wo...

> Can you assign a Bayesian probability to that?

It depends on what you mean by "that". You can assign a Bayesian probability to any proposition, but you have to be careful because not all statements are propositions. For example: "Penguins are beautiful" is not a proposition. It makes no sense to say that it is "true" or "false" because it is ambiguous (do you mean all penguins or just some penguins?) and it involves a value judgement.

The MWI is particularly thorny because it is really easy to hide ambiguity and value judgements in seemingly innocuous language. For example, you will often hear proponents of the MWI say something like, "The universe splits every time a quantum measurement is made." But that is ambiguous because it doesn't tell you what a quantum measurement is. It also has a tacit "value judgement" hidden in the phrase "the universe". The whole point of the MWI is that there is not one universe, there are unfathomably many of them. So which universe is "the" universe? Well, to me, "the" universe is the universe that "I" live in. But what does the word "I" mean in the context of the MWI? Because according to the MWI, every time the universe splits, I split along with it and become multiple me's in multiple universes. So which "me" is the real me?

The MWI is very strongly analogous to the beauty of penguins.

> I think you misinterpreted what I said. My belief in Bayesian reasoning was very nearly 1 from the first time I encountered Bayes's theorem. Nothing I've seen since has changed this, i.e. nothing about Bayes's theorem has surprised me.

My question was whether you used Bayesian reasoning to accept Bayesian reasoning as your epistemology. And I don't think you did. Instead, I think you were given a persuasive argument for why it's a good way to think about evaluating claims or making predictions, and then you started to think that way.

MWI is an example where there isn't a good way to assign a probability, because so far it's not testable. Instead, people like Sean Carrol are convinced by the mathematics that it's true, or they're skeptical for some reason like it violating Occam's razor or the measurement problem you pointed out. At any rate, it's isn't because there is some probability attached.

IOW, Bayesian reasoning isn't the only reason we believe things.

> My question was whether you used Bayesian reasoning to accept Bayesian reasoning as your epistemology. And I don't think you did.

Ah. Yes, I agree. In fact, I very rarely actually do the Bayesian math.

My core epistemology is Popperian.

> MWI is an example where there isn't a good way to assign a probability, because so far it's not testable.

It's even worse than that. One of the predictions of the MWI is that it is not testable! The MWI is "not even wrong".

Priors are not just numbers between 0 and 1. They are probability distributions. If they were not, it would be impossible to update beliefs based on evidence.

My prior beliefs about the coin include the possibility that the coin has a bias other than 50%. This distribution may be centered at 50%, but I may assign more or less probability to extreme biases, and this determines how I update my belief when I observe several heads in a row.

Of course it is possible to update beliefs based on evidence if priors are numbers. If I flip a coin but don't show you the outcome, you will rationally assign a prior of 0.5 to the proposition "The coin landed heads-up". If I then show you that the coin did indeed land heads-up (or tails-up) you will update your assessment of the probability of "The coin landed heads up" to something close to 1 (or 0).

Now, it is possible to do Bayesian analysis on probability distributions rather than "bare" priors, but that is just a way of assigning priors to a family of propositions rather than single propositions. For example, "This is a fair coin" is not a proposition to which one can assign a prior because it isn't a well-defined proposition. The intuition behind "fair coin" is something like "the ratio of heads to tails will approach 1 as the number of flips goes to infinity". But no coin can be flipped an infinite number of times, so this is a meaningless definition. Furthermore, consider a coin that always landed on the face opposite that of the previous flip. The ratio of heads to tails for such a bizarre coin would indeed approach 1 as the number of flips approached infinity, but that is not what is generally meant by a "fair coin". So the usual way of dealing with coins is assigning priors to the family of propositions, "If I flip this coin N times the probability that I will get M heads is X" for all possible values of N and M. But each of the propositions in that family gets a prior which is a number.

That framing actually stops holding for continuous distributions. Like in a distribution on heights of people there isn't a probability or prior associated with a particular height. Probabilities are assigned to subsets of this distribution (sometimes called sigma algebras). Priors stands for prior distribution and it really is best to see Bayesian analysis as a machine that consumes probability distributions as input and spits out probability distributions as output.

> That framing actually stops holding for continuous distributions.

No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number.

> Priors stands for prior distribution

No. Bayes's theorem is:

P(H|E) = P(E|H) * P(H) / P(E)

The "prior" is P(H) which is a probability, i.e. a number between 0 and 1, as are all the other "P" values. The "*" and "/" operations in Bayes's theorems are scalar multiplication and division.

> No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number.

A continuous distribution does not assign probabilities to each proposition but subsets of them. To see this concretely consider the continuous Uniform distribution from 1 to 1.5. It will for all values of its support have a probability density of 2. Most people would not consider 2 a probability.

For continuous distributions, Bayes's theorem becomes about probability densities and not probabilities.

> A continuous distribution does not assign probabilities to each proposition but subsets of them.

Nope. It assigns probabilities to individual propositions.

> Most people would not consider 2 a probability.

That's true, but in your example 2 is not a probability but a probability density, and a probability density is not the same as a probability. To get a probability out of a probability density you have to integrate. For each possible interval over which you could integrate there is a corresponding proposition whose probability of being true is exactly the value of the integral.

Ok I see where my confusion was. For you propositions are things that can be associated with measurable sets.

No, propositions are simply statements for which is it meaningful to assign a truth value. It might be possible to come up with a proposition that is associated with a non-measurable set, though I can't offhand think of an example. But remember: Bayesian probabilities are models of belief. Priors are peronsal. And so in order to assign a prior to a proposition, the statement of the proposition must have some referent in your personal ontology. You can't hold a belief about the truth value of a statement unless you know (or at least think you know) what that statement means. So unless a person has something in their ontology that corresponds to a non-measurable set (and I suspect most people don't) then that person cannot assign a Bayesian prior to a statement associated with a non-measurable set. But that's a limitation of that particular individual, not a limitation of Bayesian reasoning. For example: you cannot assign a Bayesian prior to the statement, "The frobnostication of any integer is even" because you don't know what a frobnostication is.

(Note that there are all kinds of ways that statements can fail to be propositions. For example, you can't assign a Bayesian prior to the statement: "All even integers are green" despite the fact that you know what all the words mean.)

But priors are not *just* numbers between 0 and 1, as GP claimed.

It is true that in your example, the outcome is binary: there is only a single proposition (the coin landed heads). And so the prior distribution is a Bernoulli distribution, but I would agree you could think of this prior as a single number between 0 and 1.

But in such a scenario, the only possibilities for updating beliefs based on evidence is to observe heads, then update the posterior to P(Heads)=100%, or observe tails, then update posterior to P(Heads)=0%. I might say that this is updating your beliefs based on evidence, but only trivially so.

If you observe heads and form a posterior of P(Heads)=99.999%, then it means that your prior probability distribution was more complex: it included the possibility that "the coin landed heads but I mistakenly observed tails". In this case, your priors include a family of propositions, not a single number.

In GPs example, the prior included the possibility that the coin was magnetic, etc. So it could not be represented by a single number.

> the prior distribution is a Bernoulli distribution

No, it isn't. The prior on the outcome of multiple tosses is a Bernoulli distribution, but the prior on the outcome of a single toss is a number. And for a fair coin, that number is 0.5 before you see the result, and it is either 0 or 1 after.

Note that "This coin will land/landed heads" is a different proposition than "this coin is a fair coin." The latter is much more problematic because it is hard to define what a "fair coin" is.

> your prior probability distribution was more complex

No. You are confusing my prior on a single proposition (a number) with the set of priors on all propositions which I have considered in my lifetime. The latter is obviously a set of numbers, one for each proposition.

> "the coin landed heads but I mistakenly observed tails"

Yes, that is two separate propositions:

1. The coin landed heads.

2. My senses are reliable.

All evidence must be assessed relative to some "background model", a set of propositions whose priors are very nearly 1. And it is also possible to model an infinite family of propositions about continuous properties, e.g. The length of that rod is between X and Y for any two real numbers X and Y. But none of this changes the fact that for any single proposition the prior is a number.

> it could not be represented by a single number

That depends on what you mean by "it". The probability that the coin landed heads certainly can be represented by a single number, as can the probability that the coin is magnetic. Two different propositions, hence two different priors.

> suppose you are asked to assign a probability to the proposition that there was once life on Mars

As soon as you accept this question as meaningful, you have already come out as a Bayesian. Deutsch would simply say (if I understand him right) that this question is not a matter of probabilities, whereas the coin question is.

Suppose you were a bookie and somebody wanted to make a million dollar bet on it. You'd love to take the money, but obviously you need a line. How do you pick it?

A bookie would not care at all what the real likelihood of life on mars would be. A bookie would care about the likelihood that would attract approximately equal bets on both sides of the line, since that way they guarantee that they make money. So they would do whatever sociological research would be required to estimate that and set the line accordingly and probably not do an ounce of research on Mars itself.

This assumes that there is a chance to create a matched book. This is usually possible for something "liquid" like a big league sporting event, but not always for poorly-followed sports or weird prop bets on low-interest games. Even for liquid games, the book is not always perfectly matched and they may end up with some actual exposure to the outcome of the event.

In any case, in this scenario its just one guy looking to make a bet. There is definitely SOME line, possibly million-to-one or higher, that a smart book (or any rational actor) should be willing to take the bet.

> Predicting the next draw from a deck of cards is subject to "out of context" model violations the same way that any other prediction over future world states is.

The observation that the uncertainty is only present in the observer is certainly true in the card context, but really questionable in the world states one. For one, don't we need to assume a really aggressive "deterministic evolution of future world states" to argue this uncertainty is only present in the observer? I, for one, am totally uncomfortable with this assumption...

Also, there's a clear computational difference between these two settings. You kind of point this out by acknowledging that explicit Bayesian calculation are unreasonable in many settings - but in practice, I'm on a rationalist email thread where folks are trying to calculate explicit probabilities about the increased likelihood of nuclear war over the past 6 months. It's totally silly.

We need to look at the actual way that Bayesian tools are actually used (and usable in practice) by it's adherents. As far as I've observed, it's mostly just silly signaling games where people make up numbers to justify whatever story they want to tell (given that the rationalist communities spend so much time worrying about confirmation bias as a fundamental one, I'm not sure this is even surprising).

Also, superforcasters are most certainly not "proof that explicit Bayesian calculations are still extremely useful." There are literally _millions_ of experts who make prediction - there's no version of history where this isn't a random subset who performs dramatically better than average just due to statistical chance! Misunderstanding this as a proof of the usefulness of the probabilistic reasoning tool is the classic example of being fooled by randomness.

>a really aggressive "deterministic evolution of future world states"

What would it be if not deterministic? Chaotic sure, but that just means things are very sensitive to initial conditions. Unless we extend those initial conditions to include quantum uncertainty then things are pretty much deterministic.

Of course that idea goes out the window depending on your opinions on free will.

Even if it's entirely deterministic above the quantum/micro level, that doesn't mean we can predict anything without computing the entire future world, just like in the game of life. Also, the wavefunction is deterministic, so you could in principle compute the many worlds, but that wouldn't tell us which one we will observe.

> The observation that the uncertainty is only present in the observer is certainly true in the card context, but really questionable in the world states one. For one, don't we need to assume a really aggressive "deterministic evolution of future world states" to argue this uncertainty is only present in the observer? I, for one, am totally uncomfortable with this assumption...

My objection is to Deutsch's special pleading with regards to games of chance, or whatever else he thinks is "based on a physical understanding of the situation where a randomising process had approximated probabilities". The difference in irreducible uncertainty between "shuffled cards" and "anything else" is a matter of degree, not kind.

> Also, there's a clear computational difference between these two settings. You kind of point this out by acknowledging that explicit Bayesian calculation are unreasonable in many settings - but in practice, I'm on a rationalist email thread where folks are trying to calculate explicit probabilities about the increased likelihood of nuclear war over the past 6 months. It's totally silly.

Is it? Rationalists did better than pretty much any other set of people that could usefully be regarded as a "community" when it came to COVID, in terms of seeing it coming and dealing with it successfully. Do you think they did this by gut feeling? (Sure, some of them did, but much of that gut feeling was informed by explicitly probabilistic reasoning performed by others.)

I'd love to have a better toolset for deciding when the level of risk involved in staying in [random major city] crosses a threshold that justifies moving. Right now we have mechanistic reasoning (what effects do nuclear explosions have) and reference class forecasting (how likely is Putin to do [some unusual thing]), both of which spit out credences with respect to the questions we care about (how likely am I to die if I stay here). The point isn't to follow the numbers off a cliff, but if you're well-calibrated (as in, have a track record with a decent brier score, or something), then pretending those numbers are totally useless is a bit silly. The question is not "are these numbers wrong" - yes, of course they are! The question is what alternative you're proposing, and whether it's any better. If you aren't well-calibrated and don't have experience using explicit Bayesian reasoning to make major life decisions like "I should leave this city because I think the risk of it getting bombed in a nuclear war has crossed some threshold", then I don't suggest you start now. But you could do worse than looking at people who do have such track records. Well, you'd be able to do that in a world where we cared about keeping track of that sort of thing. Too bad people keep finding reasons not to do that, huh?

> We need to look at the actual way that Bayesian tools are actually used (and usable in practice) by it's adherents. As far as I've observed, it's mostly just silly signaling games where people make up numbers to justify whatever story they want to tell (given that the rationalist communities spend so much time worrying about confirmation bias as a fundamental one, I'm not sure this is even surprising).

Surely you aren't telling me that you've updated your priors on observed evidence, and as a result have a different expectation about future world states (that is, "how useful would explicit Bayesian reasoning be if I tried using it")?

> Also, superforcasters are most certainly not "proof that explicit Bayesian calculations are still extremely useful." There are literally _millions_ of experts who make prediction - there's no version of history where this isn't a random subset who performs dramatically better than average just due to statistical chance! Misunderstanding this as a proof of the usefulness of the probabilistic reasoning tool is the classic example of being fooled by randomness.

Superforecasters consistently perform much better than chance (about as well as domain experts who aren't superforecasters, for difficult questions, in fact). Their level of performance is not compatible with the "the small number of people out millions for whom the the coin flip came up heads 10x in a row" hypothesis - if you had conducted an experiment to distinguish between the hypotheses "Superforecasters will perform no better than chance" and "Superforecasters will perform [x] better than chance", the posterior likelihood ratio in favor of the second would be truly absurd, even if you were pretty far off on where exactly in the distribution they'd be.

Thanks for the points and taking the time to write up so much! I am enjoying this interaction, even if we don't fully agree :}

> The difference in irreducible uncertainty between "shuffled cards" and "anything else" is a matter of degree, not kind.

Difference in degree or difference in kind is not the important point to sort out. The important question is what computation methods actually work in practice for humans. Some algorithms that work well on degree-small datasets fail to terminate in the course of the universe for large-datasets.

If you've ever operated in a particular uncertain environment (like, as a first time startup founder), it immediately becomes clear how totally useless explicit probability calculations are when making decisions. I was a self-proclaimed rationalist and tried reasonable hard to be a good Baysian; I ended up deciding I might as well sacrifice a goat and read it's entrails.

> The point isn't to follow the numbers off a cliff, but if you're well-calibrated (as in, have a track record with a decent brier score, or something), then pretending those numbers are totally useless is a bit silly.

I'm actually trying to make a stronger point that these numbers are "totally useless." I think in practice many applications of explicit probability calculation are actually quite harmful.

Rationalists love to talk about map and territory as the two items to be concerned about, but there's also another thing called "agent's belief in the effectiveness of their map." What models/explicit math/probability gets you in improved mapping, it takes away from you by making one's belief in the effectiveness of their map much much more.

If you look at large-scale model failure in practice, it's almost always because someone thought the math they were doing captured the full system in a complete way (and then it didn't). 2008 is a fantastic example of this.

> Well, you'd be able to do that in a world where we cared about keeping track of that sort of thing. Too bad people keep finding reasons not to do that, huh?

I'd love to start tracking predictions generally! I agree with you here!

> Surely you aren't telling me that you've updated your priors on observed evidence, and as a result have a different expectation about future world states (that is, "how useful would explicit Bayesian reasoning be if I tried using it")?

To be clear: it's the explicit probability calculation and mathification that I take my major issue with. I am most definitely not against learning from what I observe :-)

> Their level of performance is not compatible with the "the small number of people out millions for whom the the coin flip came up heads 10x in a row" hypothesis

Can you link this math? I'd love to see it - not flippant, genuinely looking to check it out and be educated here!

Yeah, to clarify, my original objections were generated by Deutsch's dismissals of Bayesian epistemology on grounds of fundamental invalidity, rather than practicality. I agree that practicality is often a serious concern! I've also attempted a startup and not once did I ever run an explicit Bayesian calculation.

Naive applications of many powerful techniques often turn out to be actively harmful, unfortunately.

But without explicit probability calculation and mathification, we can't actually track our predictions (often of hugely uncertain events) over time, and if you can't track your predictions over time you'll have a hard time improving.

https://goodjudgment.com/resources/the-superforecasters-trac...

I don't know if anyone's written up the math, but it should be pretty intuitive to get a sense of the differential. Consider that you need 20 bits of information to distinguish one thing from approximately a million. (Isomorphic: the odds of a coin coming up heads 20 times in a row is 1/2^20.) Now, of course, superforecasters aren't perfect predictors, but they're also predicting things much harder than coin flips - there's no such thing as "playing it safe" with brier scores. Matching or outperforming domain experts across a wide variety of domains, over a long period of time, involving many questions (hundreds if not thousands) is not really the sort of thing that happens by coincidence.

> fundamental invalidity, rather than practicality

I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions.

I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions.

My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools.

To paint the shape of what I think these decision making tools should look like:

1. They avoid mechanistic explanations.

2. They are explicitly not mathy.

3. They apply simple and robust heuristics to generate lots of options.

4. They exist in a purposely constructed iterative environment so that you get lots of attempts.

These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable!

I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of).

Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!

> by acting in the world you are making predictions with numbers on them all the time!

This was a key insight for me several years ago. Pretty much any time you make a choice, you are implicitly making a probability calculation, something along the lines of "which of my options has the highest expected value/probability of success?" You may not be putting specific numbers on things, but you are necessarily making an inference about which of several numbers is largest. So the question is not whether you should be making predictions based on probability (you don't have a choice) but instead how you should be doing so.

> Pretty much any time you make a choice, you are implicitly making a probability calculation, something along the lines of "which of my options has the highest expected value/probability of success?"

That's not quite right. The calculation is more along the lines of "which of my options has the highest risk-adjusted reward?" A 10% chance of getting $1,000 is more attractive than a 50% chance of getting $100.

It should be noted though that there are other ways to model the decision, such as regret minimization and its variants.

It depends on the choice being made. If you're considering one of several routes on the way to work, you don't care about arrival time, you only care about probability of arriving before a certain time.

In any case, the calculation isn't the point. My point is that in any non-trivial choice, you are necessarily making such a calculation, even if only approximately.

>There's no fundamental ontological difference between "give me your likelihood ratios between drawing cards [a and b] as the next draw from this deck of cards" and "give me your likelihood ratios between finding and not finding a continent in an unexplored part of the world".

These are not the same, my observables being independent of each other is not the same thing as my not knowing about the correlated observable. In a practical sense, there are lots of ways that I could reason that there is a continent in the unexplored bit, and obviously no way to know which card will come up in a shuffle.

My point is that various observations appear "independent" (i.e. uncorrelated) only because of uncertainty in our map. (You could point to quantum randomness but it doesn't seem likely that this matters for most of the questions we're trying to answer, but even if it did there'd be no problem modeling it with Bayes.)

It's possible I'm misunderstanding your objection, though.

>I could reason that there is a continent in the unexplored bit, and obviously no way to know which card will come up in a shuffle.

You can reason the continent exists, but you can't know. You're using knowledge you already have. Similarly, knowledge accumulates as you run through a deck, which can be used to reason that a specific card will come up next. I'm not sure I see a difference.

Reply on news.ycombinator.com