This kind of super-cautious mindset can’t be modeled with any real valued money X (current state of the world) → utility type of mapping.
If you would trade a .99999 probability of $100M for a .99997 probability of $100B, then you’re correct—you have no consistent utility function, and hence you can be money-pumped by the Allais Paradox.
And as I’ve argued before, that only follows if the a) the subject is given an arbitrarily large number of repeats of that choice, and b) their preference for one over the other is interpreted as them writing an arbitrarily large number of option contracts trading one for the other.
If—as is the case when people actually answer the Allais problem as presented—they merely show a one-shot preference for one over the other, it does not follow that they have an inconsistent utility function, or that they can be money-pumped. When you do the experiment again and again, you get the expected value. When you don’t, you don’t.
If making the “wrong” choice when presented with two high-probability, high-payoff lottery tickets is exploitation, I don’t want to be empowered. (You can quote me on that.)
Yes, but I can’t find it at the moment—it came up later, and apparently people do get money-pumped even on repeated versions. The point about what inferences you can draw from a one-shot stands though.
If making the “wrong” choice when presented with two high-probability, high-payoff lottery tickets is exploitation, I don’t want to be empowered. (You can quote me on that.)
Not very quotable, but I may be tempted to do so anyway.
Aw, come on! Don’t you see? “If X is wrong, I don’t want to be right”, but then using exploitation and empowerment as the opposites instead?
Anyway, do you get the general point about how the money pump only manifests in multiple trials over the same person, which weren’t studied in the experiments, and how Eliezer_Yudkowsky’s argument subtly equates a one-time preference with a many-time preference for writing lots of option contracts?
The above example had no consistent (real valued) utility function regardless off my 100M@.99999 vs. 100B@.99997 preference.
BTW, whatever would that preference be (I am a bit unsure, but I think I’d still take the 100M as not doing so would triple my chances of losing it) I did not really get the conclusion of the essay. At least I could not follow why being money-pumped (according to that definition of “money pumped”) is so undesirable from any rational point of view.
If you would trade a .99999 probability of $100M for a .99997 probability of $100B, then you’re correct—you have no consistent utility function, and hence you can be money-pumped by the Allais Paradox.
And as I’ve argued before, that only follows if the a) the subject is given an arbitrarily large number of repeats of that choice, and b) their preference for one over the other is interpreted as them writing an arbitrarily large number of option contracts trading one for the other.
If—as is the case when people actually answer the Allais problem as presented—they merely show a one-shot preference for one over the other, it does not follow that they have an inconsistent utility function, or that they can be money-pumped. When you do the experiment again and again, you get the expected value. When you don’t, you don’t.
If making the “wrong” choice when presented with two high-probability, high-payoff lottery tickets is exploitation, I don’t want to be empowered. (You can quote me on that.)
This is what I’m thinking, too. Curious, since you say you’ve argued this before, did Eliezer ever address this argument anywhere?
Yes, but I can’t find it at the moment—it came up later, and apparently people do get money-pumped even on repeated versions. The point about what inferences you can draw from a one-shot stands though.
Not very quotable, but I may be tempted to do so anyway.
Aw, come on! Don’t you see? “If X is wrong, I don’t want to be right”, but then using exploitation and empowerment as the opposites instead?
Anyway, do you get the general point about how the money pump only manifests in multiple trials over the same person, which weren’t studied in the experiments, and how Eliezer_Yudkowsky’s argument subtly equates a one-time preference with a many-time preference for writing lots of option contracts?
Yep.
Rockin.
The above example had no consistent (real valued) utility function regardless off my 100M@.99999 vs. 100B@.99997 preference.
BTW, whatever would that preference be (I am a bit unsure, but I think I’d still take the 100M as not doing so would triple my chances of losing it) I did not really get the conclusion of the essay. At least I could not follow why being money-pumped (according to that definition of “money pumped”) is so undesirable from any rational point of view.