I think the real point here (as I’ve commented elsewhere) isn’t that using point estimates is inherently a mistake, it’s that the expected value is not what we care about. They’re valid for that, but not for the thing we actually care about, which is P(N=0).
What do you mean by “real point”? Don’t you mean that the point of the paper is that someone makes a particular mistake?
I mean the mistake of computing expected number rather than probability. I guess the people in the 60s, like Drake and Sagan probably qualify. They computed an expected number of planets, because that’s what they were interested in, but were confused because they mixed it up with probability. But after Hart (1975) emphasizes the possibility that there is no life out there, people ask the right question. Most of them say things like “Maybe I was wrong about the probability of life.” That’s not the same as doing a full bayesian update, but surely it counts as not making this mistake.
It’s true that Patrick asserts this mistake. And maybe the people making vague statements of the form “maybe I was wrong” are confused, but not confused enough to make qualitatively wrong inferences.
Huh, interesting. I have to admit I’m not really familiar with the literature on this; I just inferred this from the use of point estimates. So you’re saying people recognized that the quantity to focus on was P(N>0) but used point estimates anyway? I guess what I’m saying is, if you ask “why would they do that”, I would imagine the answer to be, “because they were still thinking of the Drake equation, even though it was developed for a different purpose”. But I guess that’s not necessarily so; it could just have been out of mathematical convenience...
Definitely mathematical convenience. In many contexts people do sensitivity analysis instead of bayesian updates. It is good to phrase things as bayesian updates, if only as a different point of view, but when that is the better thing to do (which in this case I do not believe), trumpeting it as right and the other method as wrong is the worst kind of mathematical triumphalism that has destroyed modern science.
I think the real point here (as I’ve commented elsewhere) isn’t that using point estimates is inherently a mistake, it’s that the expected value is not what we care about. They’re valid for that, but not for the thing we actually care about, which is P(N=0).
I’m skeptical that anyone ever made that mistake. Can you point to an example?
The paper doesn’t claim anyone did, does it?
Made what mistake, exactly?
What do you mean by “real point”? Don’t you mean that the point of the paper is that someone makes a particular mistake?
I mean the mistake of computing expected number rather than probability. I guess the people in the 60s, like Drake and Sagan probably qualify. They computed an expected number of planets, because that’s what they were interested in, but were confused because they mixed it up with probability. But after Hart (1975) emphasizes the possibility that there is no life out there, people ask the right question. Most of them say things like “Maybe I was wrong about the probability of life.” That’s not the same as doing a full bayesian update, but surely it counts as not making this mistake.
It’s true that Patrick asserts this mistake. And maybe the people making vague statements of the form “maybe I was wrong” are confused, but not confused enough to make qualitatively wrong inferences.
Huh, interesting. I have to admit I’m not really familiar with the literature on this; I just inferred this from the use of point estimates. So you’re saying people recognized that the quantity to focus on was P(N>0) but used point estimates anyway? I guess what I’m saying is, if you ask “why would they do that”, I would imagine the answer to be, “because they were still thinking of the Drake equation, even though it was developed for a different purpose”. But I guess that’s not necessarily so; it could just have been out of mathematical convenience...
Definitely mathematical convenience. In many contexts people do sensitivity analysis instead of bayesian updates. It is good to phrase things as bayesian updates, if only as a different point of view, but when that is the better thing to do (which in this case I do not believe), trumpeting it as right and the other method as wrong is the worst kind of mathematical triumphalism that has destroyed modern science.