Can’t give details, there would be a risk of revealing my identity.
I have come up with a hypothesis to explain the inconsistency. Eliezer’s verbal estimate of how many similar claims he can make, while being wrong on average only once, is actually his best estimate of his subjective uncertainty. How he would act in relation to the lottery is his estimate influenced by the overconfidence bias. This is an interesting hypothesis because it would provide a measurement of his overconfidence. For example, which would he stop: The “Destroy the earth if God exists” lottery, or “Destroy the earth at odds of one in a trillion”? How about a quadrillion? A quintillion? A googleplex? One in Graham’s number? At some point Eliezer will have to prefer to turn off the God lottery, and comparing this to something like one in a billion, his verbal estimate, would tell us exactly how overconfident he is.
Since the inconsistency would allow Eliezer to become a money-pump, Eliezer has to admit that some irrationality must be responsible for it. I assign at least a 1% chance to the possibility that the above hypothesis is true. Given even such a chance, and given Eliezer’s work, he should come up with methods to test the hypothesis, and if it is confirmed, he should change his way of acting in order to conform with his actual best estimate of reality, rather than his overconfident estimate of reality.
Unfortunately, if the hypothesis is true, by that very fact, Eliezer is unlikely to take these steps. Determining why can be left as an exercise to the reader.
Can’t give details, there would be a risk of revealing my identity.
I have come up with a hypothesis to explain the inconsistency. Eliezer’s verbal estimate of how many similar claims he can make, while being wrong on average only once, is actually his best estimate of his subjective uncertainty. How he would act in relation to the lottery is his estimate influenced by the overconfidence bias. This is an interesting hypothesis because it would provide a measurement of his overconfidence. For example, which would he stop: The “Destroy the earth if God exists” lottery, or “Destroy the earth at odds of one in a trillion”? How about a quadrillion? A quintillion? A googleplex? One in Graham’s number? At some point Eliezer will have to prefer to turn off the God lottery, and comparing this to something like one in a billion, his verbal estimate, would tell us exactly how overconfident he is.
Since the inconsistency would allow Eliezer to become a money-pump, Eliezer has to admit that some irrationality must be responsible for it. I assign at least a 1% chance to the possibility that the above hypothesis is true. Given even such a chance, and given Eliezer’s work, he should come up with methods to test the hypothesis, and if it is confirmed, he should change his way of acting in order to conform with his actual best estimate of reality, rather than his overconfident estimate of reality.
Unfortunately, if the hypothesis is true, by that very fact, Eliezer is unlikely to take these steps. Determining why can be left as an exercise to the reader.