No need to use made-up numbers when we have real ones. In the US in 2007 there were 37,248 fatal crashes and 3.030 trillion vehicle-miles driven. (Source). That’s one fatal accident per 81.35 million miles. So, solving a Poisson distribution for P(E|H) >= 0.95, where the evidence is the number of miles driven by autonomous vehicles without a fatal accident:
λ^k * e^-λ / k! = .05; k = 0
e^-λ = .05
λ = 2.996
2.996 * 81.35E6 = 243.7 million miles required for statistical significance.
This, however, is only frequentist reasoning. I would actually be inclined to trust autonomous vehicles after considerably less testing, because I consider P(H) to be a priori quite high.
I can’t agree. AI—yes, even mundane old domain-specific AI—has all sorts of potential weird failure modes. (Not an original observation, just conveying the majority opinion of the field.)
In this instance “weird failure mode” means “incident causing many deaths at once, probable enough to be a significant risk factor but rare enough that it takes a lot more autonomous miles in much more realistic circumstances to measure who the safer driver is”.
No need to use made-up numbers when we have real ones. In the US in 2007 there were 37,248 fatal crashes and 3.030 trillion vehicle-miles driven. (Source). That’s one fatal accident per 81.35 million miles. So, solving a Poisson distribution for P(E|H) >= 0.95, where the evidence is the number of miles driven by autonomous vehicles without a fatal accident:
λ^k * e^-λ / k! = .05; k = 0
e^-λ = .05
λ = 2.996
2.996 * 81.35E6 = 243.7 million miles required for statistical significance.
This, however, is only frequentist reasoning. I would actually be inclined to trust autonomous vehicles after considerably less testing, because I consider P(H) to be a priori quite high.
I can’t agree. AI—yes, even mundane old domain-specific AI—has all sorts of potential weird failure modes. (Not an original observation, just conveying the majority opinion of the field.)
Yes, but humans also have all sorts of weird failure modes. We’re not looking for perfection here, just better than humans.
In this instance “weird failure mode” means “incident causing many deaths at once, probable enough to be a significant risk factor but rare enough that it takes a lot more autonomous miles in much more realistic circumstances to measure who the safer driver is”.
Yup, humans have weird failure modes but they don’t occur all over the country simultaneously at 3:27pm on Wednesday.