Understand statistics before employing them please. We don't have enough data and a single data point doesn't change that.
Do you understand statistics?
Similar if you drive 1 million km and kill 1 person, if I drive 10 km and kill 1 person is still 1 data point and I can make no conclusion? I think it would have been 1 data point if this was the first km a Uber self driving car has driven.
Yep. Because I still have another 9 m km to go before I've driven as long as you have and there is no way to know whether I'm going to kill another 9 people, or 0 more, until I've actually driven them all.
Also you have a mistake in your comment, I would still have to do 999990 km of driving. If I killed a person in my first 10 km what is the probability that I won't kill anyone in my next 999990?
Your point is that I can't be 100% sure and that is true but we can compute the probability, so the probability that I had bad luck is very small, if the probability of killing 1 person in 1 mil km is 1 or 100% what is the probability of killing this person in my first 10km? ( you are correct is not 0 )
To assess the risk posed by a driver, you wouldn't follow them around, constantly logging the miles they drive, counting the deaths they cause and continuously updating the probability they will kill someone, at least not in the real world (in a simulation, maybe). Instead, what you'd do is wait for enough people to have driven some reasonably significant (and completely arbitrarily chosen) distance, then count the fatal accidents per person per that distance and thereby calculate the probability of causing an accident per person per that distance. That's a far more convenient way to gather statistics, not least because if you take 1000 people who have causd an accident while driving, they'll each have driven a different distance before the accident.
So you might come up with a figure that says "Americans kill 1.18 people every million miles driven" (it's something like that, actually, if memory serves).
Given that sort of metric, you can't then use it for comparison with the performance of someone who has only driven, say, 1000 miles. Because if you did, you would be comparing apples and oranges: 1 accident per 1000 miles is not on the same scale as ~1 accident per million miles. There's still another 999k miles to go before you're in the same ballpark.
And on that scale, no, you can't know whether an accident in the first 1000 miles will be followed by another in the next 1000 miles. Your expectation is set for 1 million miles.
It's a question of granularity of the metric.
But think about this, if I killed a person in my first 10 km of driving, what is the chance that will kill 0 after the next 999990, would you bet that I will kill 0 or 1 , more then 10?
As to your question- I wouldn't bet at all. There is no way to know.
Here's a problem from me: I give you the number 345.
What are the chances that the next number I give you is going to be within 1000 numbers of 345?
I draw random numbers from 0 to Max and I get 345, what is P that next number N is in 100 range near 345?
P = 200/Max; in the assumption that Max >445;
For self driving cars, the probability that a car kills a person for 1 km or road driven is unknown, so you can call it X
Then my self driving car killed a person in first 10 km, What is the probability that a random event will happen in the first 10km from 10^9 km, is 10^(-8)
Say the self driving car would have the probability of killing N people for 10^9 km, this are random,independent events So the probability that a kill will happen in first 10km is N*10^-8,
I hope you notice my point that we can measure something, we do not need to wait for 10 or 100 people to be killed
We are not sure but we can say that is is a very small chance that I will not kill other person in my next 999 990km.
let me know if my logic is not correct, in statistics is easy to do mistakes.
It’s more like 10 miles of observation 300,000 times. Or rather an attentive human can be 50x better than average.
That's like saying: "I rolled this red d20 twenty times before I rolled a 1, whereas I rolled a 1 the first time on this blue d20, so the red d20 is obviously better and I'm rolling all my saves on it".
Or, I don't know- "I rolled three 1s on this d20 in twenty rolls so it's obviously not a fair d20".
If you have a strong prior the dice are equivalent, then yes, the rolls shouldn't change your mind.
If you have a prior that the dice are weighted in an unknown way, then yes, the rolls really should change your mind.
The expected rate for 100m miles for Uber is 33.333...
But how confident can we be? To answer that let's compute a poisson confidence interval around that rate, as in https://stats.stackexchange.com/questions/10926/how-to-calcu....
Let's see what a 95% confidence interval for 1 death in 3m miles looks like:
> poisson.test(1,conf.level = 0.95)$conf.int
[1] 0.02531781 5.57164339
attr(,"conf.level")
[1] 0.95
> 33.333333*0.02531781
[1] 0.843927
> 33.333333*5.57164339
[1] 185.7214
> poisson.test(1,conf.level = 0.9)$conf.int
[1] 0.05129329 4.74386452
attr(,"conf.level")
[1] 0.9
Now let's look at 93% CI:
> poisson.test(1,conf.level = 0.93)$conf.int
[1] 0.03562718 5.17251332
attr(,"conf.level")
[1] 0.93
So we can conclude with 93% certainty from this data that Uber is less safe than sober drivers. Probably a LOT less safe. Although the CI is really wide, this is shocking data for Uber, in my opinion.
People with more knowledge about statistics than me might be able to explain why.
But the poster did something different. He took it one level further and attempted to calculate this confidence number for different spans in which the actual "deaths per 100 million miles" number of Uber's current cars would fall into, given an ideal world (from a data perspective) in which they would have driven an infinite amount of miles. But he actually did it the other way round - he modified the confidence variable and calculated the spans, and then he adjusted the confidence until he arrived at a span that would put Uber's cars just on par with human driving in the best case.
The fact that a fatal incident happens that early (at 3 million, and not closer or past the 86 million that a statistical human drives on average until a fatal incident occurs) does not allow us to extrapolate a sound number per 100 million miles, but it tells us something about the probability by which the actual number of fatalities by 100 million miles that we'd get if Uber continued testing just like it did and racked up enough miles (and killed people) for a statistically sound calculation will fall into different margins. Sure, Uber could have been just very, very unlucky - but that's pretty unlikely, and the unlikeliness of Uber's bad luck (and conversely the likeliness of the fact that Uber's tech is just systematically deadly) is precisely what can be calculated with this single incident.
The assumption of using Poisson seems pretty solid to me, given we are talking about x events in some continuum (miles traveled in this case), but always happy to hear any cogent objections.
The bigger problem though is what you are doing with your confidence interval. A CI is a statement about replication. A 95% confidence level means that in 100 replications of the experiment using similar data, 5 of the generated CIs -- which will all have different endpoints -- will _not_ contain the population parameter, although IIRC this math is more complicated in practice, meaning that the error rate is actually higher. As such, if you generate a CI and multiply the endpoints by some constant, that's a complete violation of what is being expressed: there is vastly more data with 100m driving miles than 3m miles, which will cause the CI to shrink and the estimate of the parameter to become more accurate. There is absolutely no basis for multiplying the endpoints of a CI!
Ultimately, given that the size of the sample has an effect on CI width, you need to conduct an appropriate statistical test to compare the estimated parameters between the 1 in 3m deaths for Uber and whatever data generated the 1.18 in 100m deaths for sober drivers. There's a lot more that needs to be taken into account here than what a simple Poisson test can do.
For an analysis of how AVs with various safety levels perform in terms of lives saved over time, I recommend https://www.rand.org/blog/articles/2017/11/why-waiting-for-p...
Edit: Note the default values of the T and r parameters when you run poisson.test(1, conf.level = 0.95), and also that the p-value of the one-sample exact test you performed is 1. Also, since this is an exact test, the rate of rejecting true null hypotheses at 0.95 is 0.05, but given my reservations about the use of a Poisson distribution here, I don't think that using an exact Poisson test is appropriate.
Null hypothesis: The true rate of events is 1 (r) with a time base of 1 (T).
Alternative hypothesis: The true rate is not equal to 1.
The reason that you end up with a p-value of 1 is because you've said that you've observed 1 event in a time base of 1 with a hypothesized rate of 1. So given this data, of course the probability of observing a rate equal to or more extreme than 1 is 1! As such, you're not actually testing anything about the data that you claim you are testing.
I'm not trying to be harsh here, but please be careful when using statistics!
> poisson.test(c(1, 11800), c(3, 1000000), alternative = c("two.sided"),conf.level = .93)
Comparison of Poisson rates
data: c(1, 11800) time base: c(3, 1e+06)
count1 = 1, expected count1 = 0.035403, p-value = 0.03478
alternative hypothesis: true rate ratio is not equal to 1
93 percent confidence interval:
1.006334 146.142032
sample estimates:
rate ratio
28.24859
Interestingly, if you multiply the CI I claimed before by the rate ratio instead of the expected rate, you get almost exactly the same CI as here.
> ci <- c(0.03562718, 5.17251332)
> 28.24859 * ci
[1] 1.006418 146.116208
> Uber is less safe than sober drivers
But the 1.18 deaths in 100m miles is for all drivers, not just the subset of sober drivers. Not quite sure why you are claiming it is only sober drivers.