You wait 5 sec and it's still only imperceptibly closer. You realize there is no way it could possibly hit you. You cross the street unconcerned.
Consider a 6-sided dice roll. What is the chance it will roll a 1?
A person might think, "1 in 6". But what if this is a loaded die? In that case, we need more information before we can classify it as "a die like other dice". We can observe two rolls, and try to ascertain whether or not it is like other fair 6-sided dice; however, two rolls is not enough to be sure.
So as we're gathering data, we start to classify this instance of a thing (a die, an asteroid) as part of a series of things we already know about. The more rolls we observe, the more sure we can be that this is a fair die or a loaded die, for instance.
If I'm understanding how asteroids' trajectories are calculated, we can simulate THIS asteroid's trajectory (3% chance of hitting you, based on a little data), or we can just decide to classify it (perhaps prematurely?) in the series "an asteroid like every other asteroid that we've observed" and arrive at a 0.000001% chance of hitting you (I'm making up a number here).
You have one confidence margin for a single single measurement and a different confidence margin if you make 1 million measurements.
Let's say you can measure marble diameters and your tool has a calibrated standard deviation of 1 mm.
If you pull one marble and measure it to be 10 mm larger than expected, you can calculate the chance you are wrong using only the standard deviation of your measurement tool.
However, if you pull 1 million marbles and measure one to be 10 mm larger than expected, you need to take into account the number of marbles you have measured.
A 3% chance that never occurs is an inaccurate prediction.
It's wrong because the measurements are suggestive of possibility, rather than certain of it.
If we observe an asteroid that with two poor measurements is determined to be headed away from Earth, that's the end. Look no further.
If we observe an asteroid with two poor measurements that has some significant chance of hitting, more and better measurements are made. Then very often those better measurements show it was never actually going to hit anyhow.
But we never would have known without the better measurements, and we never would have devoted more time to making better measurements without a reason to do so.
A 3% chance that never occurs is because that 3% is based on data that's at the limit of what the telescopes can provide, not based upon bad math.
Hence it seems that it would lead to more accurate predictions if the measurements and their uncertainties were fitted to a model that corrects for the prior probability of observing an asteroid on a given trajectory/making a certain observation.
This discrepancy between distribution of measurement error vs distribution of actual trajectories is what people are wondering about, because it seems interesting to know more about (e.g. "why are certain trajectories less likely?").
It's is similar to P hacking.
If you just want to argue with people, feel free. But based on how this conversation has been going it doesn't seem like you want to learn.
I understand that calculating trajectories is difficult.
If someone claims something like a 3% impact probability, and they are wrong 99.999% of the time, that speaks to a methodological error in how the numbers are conveyed and or defined.
I work in medical devices and testing. I perform tests like X percentage of patients will die based on the statistical calculations. You may undergo treatment with a medical device that I have worked on.
Wait a week and get another sample and your arc is now approx 5x as long. Wait a month and get another and now your arc is 30x as long as the original. More observations shrink your error bars.
There are systemic errors here for sure. Two kinds, really:
1. Limits of resolution of telescopes 2. Short sample lengths
You absolutely can't do anything about error type 1. You can fix 2 by getting more data. But there's no point in getting data on asteroids that have absolutely no possibility of hitting. So only asteroids that have some probability with limited measurements get enough better measurements that are high quality in order to find out where they're really headed.
All of these measurements of trajectories are completely uncorrelated, so you can't use the priors to adjust probabilities. I mean you can do whatever you want, but we haven't been hit by a big asteroid yet since we've had telescopes and tracking databases.
If we made adjustments based on priors we'd have to discount all collisions down to 0 irrespective of the trajectories. Seems absurd, so there must be something else going on here.
Illustrate the point, imagine a pass/fail AIDs test with 99% accuracy and 1% false positive. If you test one patient only and they are positive, You can conclude that is 99% likely to be correct. However, if you test a hundred different people and one of them comes up positive, you can no longer claim the 99% certainty for that patient. You know that you administered a hundred different tests to different people and would have to reduce your confidence accordingly because you expect one false positive. This second statistical approach is what is not happening with the asteroids, and why asteroids with a 3% chance of hitting Earth suspiciously get revised down to zero more than 97% of the time.
>If we made adjustments based on priors we'd have to discount all collisions down to 0 irrespective of the trajectories. Seems absurd, so there must be something else going on here
Not quite true. If you measure a million asteroids in the data from one says it has a trajectory towards Earth, you need to Discount that observation by the fact that you made 1 million different measurements. The outlier might still be close to zero statistically, but it did have a outlier data. This would be a reason to remeasure the asteroid multiple times. It is only through that process that the number will climb from zero, or stay at zero.
It's not that you're applying the prior that we have never observed Earth colliding asteroid. You're simply accounting for the fact that with the error bars on your measurement system, you expect one false positive in 1 million measurements.
My inference is that the 3% number we are talking about for this specific asteroid what's not calculated using the proper statistical treatment, and that's why it wasn't published in the first place.
This is also why it is similar to P hacking. If you run 20 experiments and analyze them as if they were the only experiment you did, you will get one of them that says a wrong result with 95% confidence, which is the common threshold for publishing outside of physics.
When it gets a little closer, you can tell at least which half it's on, the left or the right. Now your estimate is either 0% or 50%.
Closer still and you tell which lane it's in, so now you're sure.
What wouldn't make sense is if you repeat this 1000 times and a car is never in your lane.
That means that something is wrong about how you are modeling the road and cars.
The claim that people are confused by is (asteroids with a 3% chance of hitting get the change revised to 0% more than 97% of the time).
That's exactly it. And at the speeds these objects are going and the uncertainty of the observations you would have to be observing an object for a really long time to get the kind of accuracy required to pick a mitigation method that would work. And even then, assuming you could nail the point of impact of something going 2000 km / second of unknown mass in a strong gravity field: given the COVID response I have a hard time believing that the response to 'Houston, Texas is going to be obliterated on Jun 1st 2024' would be met with anything but skepticism and laughter. Right up to the moment of impact.