Bayesian methods to provide probablistic solution for the Drake equation (2019) | Hacker News Reader

Bayesian methods to provide probablistic solution for the Drake equation (2019)

sciencedirect.com

49 pointsbenbreen49 comments

Interesting to see this show up. This was a very formative paper for me. If correct—and it certainly seems correct to me—then it completely dissolves the Fermi paradox. And yet nobody seems to be aware of it, and both the popular and scientific press continue with the “where are they?” Fermi paradox headlines.

What I learned was that just figuring out the right answer is indifferent to change people’s minds, even in science.

What was the conclusion of this paper? The link's conclusion was truncated.

It's in the abstrct, right at the top

You could have just stayed silent.

Why, was there a joke I ruined or something?

It says only

>That said, the results indicate that the probability we are alone (<1) in the galaxy is significant, while the maximum number of contemporary civilizations might be as few as a thousand.

That's not very useful.

The full paper is available on scihub though.

Err, what exactly were you expecting, their telephone numbers? You got figures and probabilities in the conclusion and I can't see what else you could reasonably hope for.

I (and I believe jwocky as well) expected some kind of rationale. There are no figures nor probabilities supporting the conclusion in the linked article.

Of course if one pirates the paper they can learn more.

Ah, the 'method' part. Gotcha.

You might email the author and ask for a copy. They're often surprisingly willing to hand out individual copies.

> What I learned was that just figuring out the right answer is indifferent to change people’s minds, even in science

Seems too strong a conclusion.

The Drake equation does let you model numbers you think you know with some certainty, numbers that known to be very large, and some known unknowns. But even leaving aside unknown unknowns, the modeling of some known unknowns can boil to making models that assume numbers that are very small but (obviously) not zero because we're sitting here.

> And yet nobody seems to be aware of it, and both the popular and scientific press continue with the “where are they?” Fermi paradox headlines.

A reasonable explanation here is that the paper is not correct because it relies on unfounded priors, which is generally where most Bayesian work falls flat.

So your prior is that Bayesian papers are likely to have unfounded priors. Now I'm wondering how well-founded your prior is.

You have it backwards- the traditional Drake equation relies on unfounded priors, and by using a Bayesian approach they have avoided that problem. Instead of making up terms from nothing like the Drake equation, they are able to represent only the data we actually have, and leave the rest uncertain.

The priors are carefully encoding the actual information they have, and incorporating the extreme lack of prior knowledge as uniform or nearly uniform priors over an extremely wide range for the terms we have no data on.

That is the basic takeaway here- with almost no knowledge about a large number of factors (as the Drake equation is constructed), there is an extremely high chance that once of those unknown factors is actually nearly zero, even when your expectation value for each (e.g. what would have been used in the traditional Drake equation) is relatively high. N (number of civilizations) therefore approaches zero, even if there is no single term that you are pretty sure is near zero.

The Bayesian approach here allows for a rigorous representation of our (extreme lack of) knowledge and gets to the truth of the matter: civilizations face a huge number of possible bottlenecks, each of which we know almost nothing about the probabilities of. This means, there is a strong chance at least one of those is a massive filter, even if we don't know which.

While I agree with both the approach and result intuitively, the assumption of uniform unknown priors feels like it could be a huge source of errors

They are demonstrating a fundamental flaw in the logical reasoning behind the original Drake equation, that is robust to specific choices of distributions, or parameters to include or exclude.

Anytime you multiply a large number of uncertain probability distributions, the resulting posterior will have most of the probability mass near zero. This result is not sensitive to which distribution or bounds you choose. The Drake equation is nonsense, because it is effectively assuming certainty about every single term- and that is the only way to produce a result much higher than zero.

When you are multiplying seven unknown together, you can be fairly certain that the result is close to zero without knowing the value of any of the terms, unless you have some real information that none of terms can be near zero.

This is some dumb second order Bayesian reasoning. You’re declaring a prior for arbitrary random variables as if their distributions themselves are sampled from a distribution. They are not.

You cannot be certain that seven random things multiplied together is close to zero. That statement is very obviously wrong.

Further “near zero” is a misleading term at best because it neglects to mention that we are multiplying it by a large number to get an expected value.

Distributions are sampled from distributions -- it is this problem which makes global scepticism an even minimally interesting problem.

When faced with "global, recursive" epistemic problems one arrives at an extremely power-law asymmetric distribution where the "bayesian value" of almost all evidence is near zero.

We live our entire lives in this "nero zero" range, and i'd suppose, this makes a "pure bayesian" solution to the problem of knowledge deficient. Since we succeed in knowing, so we succeed in making hyperfine determinations.

This sort of "hyperfine epistemology" works globally to allow us to "know at all", but as you're sensing here -- it's pretty much useless for any local problem.

Perhaps this is just the single up-side of the bayesian approach to the drake eqn: it shows how impossible it is to state such an eqn, let alone evaluate it. We cannot, a priori, make such hyperfine determiniations on such circumstantial matters.

This post is full of fancy word nonsense.

“Distributions are sampled from distributions” is meaningless because you cannot define the meta distribution. But more importantly, the Drake equation is not a RANDOM SAMPLE from a population of distributions. So the idea of sampling distributions is irrelevant even if true. The naive math of multiplying them together is invalid.

It's always fun to see frequentists/bayesians infights in HN comments. It's almost a guilty pleasure for me.

Everyone in this thread appears to prefer Bayesian

There's the bonus problem of: even if you magically have correct priors, you still need to assume that Drake's Equation is a good model for the generation process of civilizations. If the equation is missing terms or has extra terms, no amount of Bayesian reasoning helps correct for that.

It's like thinking that you can use Bayesian reasoning to determine the likelihood of Russell's Teapot existing.

Yes, this is essentially what i mean by "distributions are sampled from distributions", ie., there's a subjective uncertainty in the choice of model but also an uncertainty in the very determination of that uncertainty.

You can model a plausible "final stable distribution", after all these recursions, with a power law.

This makes intuitive sense, if you consider how science works: all the confirmatory evidence in the world doesn't help, all the information lies in the single refutative point. This is how power laws work.

So basically we're always operating under a heavily under-determined region with high uncertainty, and we can only improve that by disconfirmatory apparent outliers.

It really doesnt matter what the "meta-distribution" is, bar trivial ones. My whole point is that we can augment bayesianism by a-priori choosing these meta-distrbutins.

Is my hand in front of me? Is what's in front of me real? Are my perceptions indicative of reality? etc. -- keep recursing

If you're drawing from any sort of epistemically plausible (ie., any plausible model of subjective uncertaintiy) distribution on each of these points, you'll "recurse" to some extreme distribution --- where all possible evidence basically makes no difference.

This is why there cant really be an "evidential" case for realism --- and why bayesianism is an incomplete epistemology.

You have to assert the truth of some basic facts, and thereby focus in on a "region" of this "extreme distribution" which is near-zero. And say only, "simply by being above zero, i'll believe it".

That's the solution to the problem of scepticism.

But this doesn't work for local issues, because locally there really isnt any kind of non-bayesian a-priori analysis which can say, "here, believe the non-zero".

ie., you can 'complete' bayesianism globally by meta-theoretical concerns, but not locally. Meaning that 'from ignorance, only ignorance' everywhere, esp. the drake eqn.

The failure of bayesianism is an indictment of darke-like reasoning -- this only works on genuienly global matters.

eg., "a priori, the world exists, therefore the meta-distributin must be so constrainted..."

Do you read your own writing here?

I'm talking to a very rarified audience, for sure. Giving a bayesian gloss on moorean epistemology is not really a project for a hacker news comment.

Please try to keep the discourse civil. You have not understood my argument. What you are dismissing as obviously wrong is a well known mathematical fact [1].

Imagine the “space” of all currently unobserved phenomena that require a series of independent hurdles to be overcome a la the drake equation: observable aliens, etc. This space is infinitely big and the probability of each of the hypothetical phenomena is astronomically low as to not even be worth considering. The ones that are worth considering have some evidence that either they have occurred, or that we think we understand the process by which they come about, and all of the series of independent hurdles are likely to be nonzero.

Imagine this simple test: take a random sample of 7+ numbers on [0,1] and multiply them together as the Drake equation does. Repeat this thousands of times to plot a smooth density plot, and you will get a stretched exponential distribution, with the majority of the probability density near zero.

This type of causal process with a cascade of independent filters multiplied together that leads to a stretched exponential is common in a lot of domains[1], and almost always makes positive outcomes very rare. For example, the probability that some random new organic molecule will bind to a specific protein target to be an effective drug is similar in this way, and is close to zero. For a molecule to work as a drug it has to pass a lot of hurdles just as a civilization does in the Drake model: be bioavailable, bind to the right target in the right way, not bind to harmful targets, be metabolized at a reasonable rate, etc.

[1] https://link.springer.com/article/10.1007/s100510050276

No, this is nonsense.

You’re implying that there are many things that could go wrong, and that if we took a random sample of “things” that we would probably find some joint distribution that is small. This is true in the sense that an incalculably small proportion of conceivable things happen.

But this particular thing is not a random sample of things. You don’t get to appeal to the unknown distribution of distributions. Your claim that it “ almost always makes positive outcomes very rare” is completely irrelevant to non randomly chosen and defined processes.

You can’t insert steps into a Bayesian inference until your priors match a desired outcome. It’s as fallacious as inserting an infinite number of steps that are highly likely but technically possible to not be the case as a way to reduce any given prior from basically guaranteed to basically never expected to happen.

Your argument reduces to “I don’t know what decides the probability of alien life but I think the chance is small” which is a fine opinion, but your mathemagics have not strengthened your argument.

Suppose we play a game called “the four game” in which case you have to guess if I’m thinking of the number 4. By your reasoning you would probably guess 0% because you don’t know the rules of generation and there are infinite conceivable ways I could draw numbers from and 1 out of infinite draws will be exactly 4. But when I play the four game I always think of 4. The imagined sampling of unknown distributions is irrelevant because the game itself is not random.

Now, look, the Drake equation tries to do it all, and that’s probably bad. Let’s re imagine it as a function that simply identifies a probability of alien life, by now, on a randomly chosen planet, multiplied by the total number of planets out there right now (let’s ignore the detectable part). Is the probability part really small? Well yeah almost certainly. No math required. Is it small enough that we can provide any confidence on the order of magnitude of the expected value? Nope. The only interesting question, imo, is whether that expected value is greater than or less than one.

The drug example you have provided is a case of exploring a new space. This is not comparable to life appearing on a planet. Because like a novel drug development, we have an example of life originating on a planet. It’s not a random sample either but it’s sufficient to observe that the process to make this happen exists and has happened. Unlike the drug development which largely a test of whether the pattern exists or not.

My previous explanation does apply to the Drake equation and all similarly constructed models because of its structure and lack of information.

However, I would agree that there isn’t much support for using the Drake equation as a model for the probability of observing alien life- it assumes too much.

It is, and that’s what the paper is about. The Drake equation folks aren’t accounting for that huge source of errors. When you do a proper accounting, as the paper does, the number of civilizations approaches N=1 and the paradox dissolves.

What I learned was that just figuring out the right answer is indifferent to change people’s minds, even in science.

The further from experimental confirmation some question is, the bigger chance that people will ignore evidence in favour of their pet belief.

There's a kind of people that likes to believe that we're too far away from our nearest neighbors, that we'll never spread to other stars, nobody will come and that we'll get extinct soon.

Not sure why they think this way. What I do know is that somehow they find comfort believing that.

Fermi paradox is music to their ears because it seems to confirm their mindset.

It's ripe for YouTubers cum science communicators to jump on.

> yet nobody seems to be aware of it

I'm a rare earthier. But I've never found the Drake equation useful for estimation. The knowns are drowned by the unknowns, which we do not know to even vast orders of magnitude. The probability of abiogenesis or multicellular life forming are simply beyond our present understanding of biochemistry. We know they are rare. But the entire equation turns on whether they are merely rare, or vanishingly so.

"With so few concurrent civilizations, and such large distances, it is little surprise that the SETI project has not found that alien signal. Our nearest neighbor is 4 light years away, and there are under 100 stars within 50 light years, the total of the project's existence."

That doesn't change Enrico Fermi's calculations on how long it might take an alien civilization to colonize the galaxy (max 100 million years). Or spam every solar system with Von Neumann probes. Or build giant radio emitters and send them out to various parts of the galaxy to ensure coverage. As long as there's a decent chance other alien civilizations have emerged before our own in the Milky Way, there is a paradox until we know why they aren't detectable.

Or seeing Type 3 civilizations in other galaxies.

>This was a very formative paper for me. If correct—and it certainly seems correct to me—then it completely dissolves the Fermi paradox.

Could you please explain it then? Make the argument that dissolves FP.

The paper shows that when multiplying seven unknown numbers together, the expected result is much, much less than multiplying seven fixed estimates together.

Sagan et al come up with a long sequence of reasonable sounding estimates that when multiplied together result in, say, N=10000, that is ten thousand advanced civilizations in our own galaxy alone. But each one of the terms involved in that calculation was highly uncertain. This paper shows that if you multiply the probability distribution for N_a and N_b together and then sample, you get a value less than sampling N_a and N_b separately and then multiplying, as Drake, Sagan, et al do.

If you perform the calculation properly using their very same estimated terms, but multiplying probability distributions instead of pre-sampling, you end up with an aggregate N value very close to 1. Meaning we should expect to be alone, and therefore no paradox.

The takeaway:

"That said, the results indicate that the probability we are alone (<1) in the galaxy is significant, while the maximum number of contemporary civilizations might be as few as a thousand."

A thousand civs spread across a galaxy means there is a low probability of meeting live aliens, but does that mean xenoarchaeology could be a thing?

Galaxy is impractically huge. 3-dimensions is infinitely bigger than 2

> Galaxy is impractically huge.

Yes, but also no... There are about 4x10^11 stars in the Milky Way galaxy, which from an exploration standpoint is massive. But 10^11 isn't exactly an inconceivably large number... let's suppose for the sake of argument that the development of complex multicellular life capable of creating and using radios is 'gated' by only two independent one in a million chances per star.. that's already 1 in 10^12. For 4x10^11 stars in a galaxy, that would mean the galaxy has a 2/3rds chance of not having even one star system with radio-capable life.

(1 - 1/10^12) ^ (4x10^11) = 0.67

> but does that mean xenoarchaeology

My favorite "crazy theory" is that the Paleocene–Eocene Thermal Maximum (PETM) was caused by a previous industrial civilization that, like us, found a bunch of easy to access hydrocarbons, naively dug them up and reached the same climate change problem that we did, ultimately wiping them out.

Of course there is 0 evidence for this at all, but it was also more than 50 million years ago. If you look into it basically nothing about an advanced industrial civilization would likely survive to make it into the geological record for us to observe. The only reason to even entertain this hypothesis is that the PETM also experience a very rapid increase in atmospheric CO2 and we aren't entirely sure why.

Unfortunately this means that there's probably not much chance for a rich field of xenoarchaeology to exist since it's not even possible to do this on our own planet.

What is an interesting thought experiment is: Suppose we realized we as a civilization were doomed and wanted to sent a message to future industrial civilization on Earth warning them about being too aggressive with hydrocarbon usage. To my knowledge there is no known method to ensure a message could be sent that far in the future, but it's fun to try to think of ways we could send a message to a future, essentially, alien civilization here on Earth.

Why would they have left no traces? Dinosaurs lived ~200 M years ago and left enough traces for us to discover them. Wouldn't an industrial civilization 50 mya have left some kind of refined metallic artifacts? Even if _most_ traces are eroded away with time it seems difficult to imagine every trace of a global industrial civilization would disappear.

There also don't appear to have been any spikes in atmospheric carbon dioxide during the relevant period: https://en.wikipedia.org/wiki/Carbon_dioxide_in_Earth%27s_at...

Though maybe you were looking at a data source with better resolution?

The answer is that the traces are not super easy to find, depending on how we characterize what traces are in the first place. The question posed is known as the "Silurian Hypothesis":

https://www.scientificamerican.com/article/could-an-industri...

> Unfortunately this means that there's probably not much chance for a rich field of xenoarchaeology to exist since it's not even possible to do this on our own planet.

I think you've reached this statement too eagerly but would be interested to discuss this point. Do you mean that the materials they used would have disintegrated and their (presumably carbon-based lifeform) bodies wouldn't have left any fossils?

My main thoughts on this come from reading Vernor Vinge's [1] excellent Marooned in Realtime which discusses some of the condundrums resulting from transferring information over massive periods of time. I think part of it talks about subduction zones where everything is eventually riven back into the Earth's mantle, essentially lost to any kind of current archaelogical techniques.

[1] https://en.wikipedia.org/wiki/Marooned_in_Realtime

Good point! The longest-lived artifacts would probably have to be somewhere with much less weather and geological activity than the surface of the earth. Maybe if you put a structure in orbit or perhaps on the moon?

Paradoxically, extending on your thought - I guess anywhere with living conditions similar to earth (oxygen atmosphere, water, weather, etc) would not be conducive to long term archival of civilization artifacts.

So you'd have to look at 'dead' places if you want to find evidence of civilization. Or stumble across an derelict megastructure (dyson sphere\niven ring\etc)

Many surfaces of the Earth are eroded away over that timescale, but not all. Otherwise we would be be able to dig up dinosaur bones.

Fermi's calculation was that it would take a civilization between and 1 and 100 million years to colonize a galaxy at sub-light speeds. There has been ample time for multiple civilizations to have done this. So where is everyone?

If there's been on average a thousand civs spread across the Milky Way for the past 5 billion years, then we need something else to explain why none of them have colonized the galaxy.

We don't even know if there is life on the moons of planets in our solar system. If we find life on Europa, Enceladus, Ganymede or any of a handful of other potential relatively local moons then the odds of life being rampant in the galaxy are pretty high. We just haven't reached the technological level to know one way or another.

The paper seems to be paywalled so I have no clue about how they arrived here, but this: "That said, the results indicate that the probability we are alone (<1) in the galaxy is significant, while the maximum number of contemporary civilizations might be as few as a thousand."

Doesn't seem to really answer anything. Isn't this just a really fancy way of saying "we don't know the solution to the Drake equation"? It could be a 99.999999...% (<100%) chance of being alone, it could be a 0% chance, or anything in between.

Given the title of the paper, this is a very loose definition of "solution" for the Drake equation ("it could be anything!").

It shows that people doing Drake equation estimates we’re doing their math wrong, and using their own numbers but accounting for uncertainty correctly you get estimates much closer to N=1.

Reply on news.ycombinator.com