I’ve been doing some more reading on DA, and I now believe that the definitive argument against it was given by Dennis Dieks in his 2007 paper “Reasoning about the future: Doom and Beauty”. See sections 3 and 4. The paper is available at http://www.jstor.org/stable/27653528 or, in preprint form, at http://www.cl.cam.ac.uk/~rf10/doomrev.pdf Dieks shows that a consistent application of DA, in which you use the argument that you are equally likely to be any human who will ever live, requires you to first adjust the prior for doom that you would have used (knowing that you live now). Then, inserting the adjusted prior into the usual DA formula simply gives back your original prior! Brilliant, and (to me) utterly convincing.
Thanks for the link. Does it work on toy models of DA in other domains?
For example, if I ask your age and you will say “30 years old” (guessing), I can conclude from it that medium human life expectancy is around several decades years with 50 per cent confidence, and that it is less than 1000 years with 95 per cent confidence.
For a general analysis along the same lines of life expectancies of various phenomena, see Carl Caves, “Predicting future duration from present age: Revisiting a critical assessment of Gott’s rule”, http://arxiv.org/abs/0806.3538 . Caves shows (like Dieks) that the original priors are the correct ones. In my example of the biologist and the the bacterium, the biologist is correct.
I know the paper, read it and found a mistake. The mistake is that while illustrating his disproval of DA, he creates special non random case, something like 1 month child for estimation of median life expectancy. It means that he don’t understand the main idea of DA logic, that is we should use one random sample to estimate total set size.
Why is that case “non random”? A randomly selected person could well turn out to be a 1 month old child. If you know in advance that this is not typical, then you already know something about median life expectancy, and that is what you are using to make your estimate, not the age of the selected person.
Do you have a criticism of Caves’ detailed mathematical analysis? It seems definitive to me.
And: to the person who keeps downvoting me. Are you treating my “arguments as soldiers”, or do you have a rational argument of your own to offer?
In Cave case this argument was nonrandom, because he knew that median life expectancy was 80 years, and deliberately choose extremely young person. So he was cheating. If he would need really random person he should apply to him self or reader. It will be real experiment.
I decided not to search math errors in his calculations, because I don’t agree this his notion of “randomness”.
I don’t see how this is cheating. Cave’s central claim is this (paraphrased rather than quoted): “If you wish to predict how long something will last on the basis of how long it has existed so far, and you have any further information about relevant time scales, then the DA will give bad predictions because it implicitly uses a prior that’s invariant under temporal scaling.”
He agrees that if you take a thousand random people and proclaim that half of them are in the first halves of their lives, you will probably be about right. But he disagrees with any version of the DA that says that for each of those people you should assign a 0.5 probability that they’re in the first half of their life—because you have some further information about human lifespans that you should be taking into account.
Cases in which the DA is applied usually have vaguer information about relevant timescales; e.g., if you want to predict how long the US will continue to exist as a nation, there are all kinds of relevant facts but none of them quite takes the form “we have a huge sample of nations similar to the US, and here’s how their lifetimes were distributed”. But usually there are some grounds for thinking some lifetimes more credible than others in advance of discovering how long the thing has lasted so far (e.g., even if you had no idea when the US came into existence you would be pretty surprised to find it lasting less than a week or more than a million years). And, says Caves, in that situation your posterior distribution for the total lifespan (after discovering how long the thing has existed so far) should not be the one provided by the DA.
So the examples he should be looking at are exactly ones where you have some prior information about lifespan; and the divergence between the “correct” posterior and the DA posterior, if Caves is right, should be greatest for examples whose current age is quite different from half the typical lifespan. So how’s it cheating to look at such examples?
Basically, there is nothing do disagree here: DA is working, but gives weaker predictions, than actual information about distributions. That is why we should try to use DA in domains where we don’t have initial distribution, just to get order of magnitude estimation.
The more interesting question is how to combine situations where we have some incomplete information about actual distribution and the age of one random object. It seems that Caves suggest to ignore DA in this case.
(But there is also Carter’s approach to DA, where DA inference is used to update information about known future x-risks, based on fact that we are before it.)
In some cases DA may be stronger than incomplete information provided by other sources. For example, if one extraterrestrial knows for sure, that any mammal life expectancy is less than 1 million years, and than he finds one human being with age 60 years, DA gives him that medium human life expectancy is less than 1000 years. In this case DA is much stronger than prior.
There’s a pretty good level of rational discussion here, better than in most online fora I know of.
Some people are pretty trigger-happy with the downvotes (and some get so worked up over political disagreements that they will go back and downvote many random unrelated comments of yours from the past—but that’s near-universally agreed to be bad and tends to get them banned eventually) but nowhere’s perfect. And I think the majority of downvotes are genuinely deserved, though I can’t see how anything in this thread deserves them.
Look, some one may say: “A fair coin could fail heads 20 times in row and you will win million dollar”. And it is true. But this does not disprove more general statement that: “playing coin for money has zero expected money win”.
The same situation is this Caves and DA.
We could imagine situation there DA is wrong, but its is true in most situations (where it is applyable)
I’ve been doing some more reading on DA, and I now believe that the definitive argument against it was given by Dennis Dieks in his 2007 paper “Reasoning about the future: Doom and Beauty”. See sections 3 and 4. The paper is available at http://www.jstor.org/stable/27653528 or, in preprint form, at http://www.cl.cam.ac.uk/~rf10/doomrev.pdf Dieks shows that a consistent application of DA, in which you use the argument that you are equally likely to be any human who will ever live, requires you to first adjust the prior for doom that you would have used (knowing that you live now). Then, inserting the adjusted prior into the usual DA formula simply gives back your original prior! Brilliant, and (to me) utterly convincing.
Thanks for the link. Does it work on toy models of DA in other domains?
For example, if I ask your age and you will say “30 years old” (guessing), I can conclude from it that medium human life expectancy is around several decades years with 50 per cent confidence, and that it is less than 1000 years with 95 per cent confidence.
Which priors I am using here?
For a general analysis along the same lines of life expectancies of various phenomena, see Carl Caves, “Predicting future duration from present age: Revisiting a critical assessment of Gott’s rule”, http://arxiv.org/abs/0806.3538 . Caves shows (like Dieks) that the original priors are the correct ones. In my example of the biologist and the the bacterium, the biologist is correct.
I know the paper, read it and found a mistake. The mistake is that while illustrating his disproval of DA, he creates special non random case, something like 1 month child for estimation of median life expectancy. It means that he don’t understand the main idea of DA logic, that is we should use one random sample to estimate total set size.
Why is that case “non random”? A randomly selected person could well turn out to be a 1 month old child. If you know in advance that this is not typical, then you already know something about median life expectancy, and that is what you are using to make your estimate, not the age of the selected person.
Do you have a criticism of Caves’ detailed mathematical analysis? It seems definitive to me.
And: to the person who keeps downvoting me. Are you treating my “arguments as soldiers”, or do you have a rational argument of your own to offer?
In Cave case this argument was nonrandom, because he knew that median life expectancy was 80 years, and deliberately choose extremely young person. So he was cheating. If he would need really random person he should apply to him self or reader. It will be real experiment. I decided not to search math errors in his calculations, because I don’t agree this his notion of “randomness”.
I don’t see how this is cheating. Cave’s central claim is this (paraphrased rather than quoted): “If you wish to predict how long something will last on the basis of how long it has existed so far, and you have any further information about relevant time scales, then the DA will give bad predictions because it implicitly uses a prior that’s invariant under temporal scaling.”
He agrees that if you take a thousand random people and proclaim that half of them are in the first halves of their lives, you will probably be about right. But he disagrees with any version of the DA that says that for each of those people you should assign a 0.5 probability that they’re in the first half of their life—because you have some further information about human lifespans that you should be taking into account.
Cases in which the DA is applied usually have vaguer information about relevant timescales; e.g., if you want to predict how long the US will continue to exist as a nation, there are all kinds of relevant facts but none of them quite takes the form “we have a huge sample of nations similar to the US, and here’s how their lifetimes were distributed”. But usually there are some grounds for thinking some lifetimes more credible than others in advance of discovering how long the thing has lasted so far (e.g., even if you had no idea when the US came into existence you would be pretty surprised to find it lasting less than a week or more than a million years). And, says Caves, in that situation your posterior distribution for the total lifespan (after discovering how long the thing has existed so far) should not be the one provided by the DA.
So the examples he should be looking at are exactly ones where you have some prior information about lifespan; and the divergence between the “correct” posterior and the DA posterior, if Caves is right, should be greatest for examples whose current age is quite different from half the typical lifespan. So how’s it cheating to look at such examples?
Basically, there is nothing do disagree here: DA is working, but gives weaker predictions, than actual information about distributions. That is why we should try to use DA in domains where we don’t have initial distribution, just to get order of magnitude estimation.
The more interesting question is how to combine situations where we have some incomplete information about actual distribution and the age of one random object. It seems that Caves suggest to ignore DA in this case. (But there is also Carter’s approach to DA, where DA inference is used to update information about known future x-risks, based on fact that we are before it.)
In some cases DA may be stronger than incomplete information provided by other sources. For example, if one extraterrestrial knows for sure, that any mammal life expectancy is less than 1 million years, and than he finds one human being with age 60 years, DA gives him that medium human life expectancy is less than 1000 years. In this case DA is much stronger than prior.
That’s a very good summary of Caves’ argument, thanks for providing it.
EDIT: I upvoted you, but now I see someone else has downvoted you. As with me, no reason was given.
I am new here at LW. I thought it would be a place for rational discussion. Apparently, however, this is not a universally held belief here.
There’s a pretty good level of rational discussion here, better than in most online fora I know of.
Some people are pretty trigger-happy with the downvotes (and some get so worked up over political disagreements that they will go back and downvote many random unrelated comments of yours from the past—but that’s near-universally agreed to be bad and tends to get them banned eventually) but nowhere’s perfect. And I think the majority of downvotes are genuinely deserved, though I can’t see how anything in this thread deserves them.
Look, some one may say: “A fair coin could fail heads 20 times in row and you will win million dollar”. And it is true. But this does not disprove more general statement that: “playing coin for money has zero expected money win”.
The same situation is this Caves and DA.
We could imagine situation there DA is wrong, but its is true in most situations (where it is applyable)
See also my large comment about Caves to gjm.