It seems that the axiom of independence doesn’t always hold for instrumental goals when you are playing a game.
Suppose you are playing a zero-sum game against Omega who can predict your move—either it has read your source code, or played enough games with you to predict you, including any pseudorandom number generator you have. You can make moves a or b, Omega can make moves c or d, and your payoff matrix is: c d
a 0 4
b 4 1
U(a) = 0, U(b) = 1.
Now suppose we got a fair coin that Omega cannot predict, and can add a 0.5 probability of b to each:
U(0.5 a + 0.5 b) = min(0.5*0 + 0.5*4, 0.5*4 + 0.5*1) = 2
U(0.5 b + 0.5 b) = U(b) = 1
The preferences are reversed. However, the money-pumping doesn’t work:
I have my policy switch at b. You offer me to throw a fair coin and switch it to a if the coin comes up heads, for a cost of 0.1 utility. I say yes. You throw a coin, it comes up heads, you switch to a and offer me to switch it back to b for 0.1 utility. I say no thanks.
You could say that the mistake here was measuring utilities of policies. Outcomes have utility and policies only have expected utility. VNM axioms need not hold for policies. But money is not a terminal goal! Getting money is just a policy in the game of competing for scarce resources.
I wonder if there is a way to tell if someone’s preferences over policies is irrational, without knowing the game or outcomes.
In this case the only reason the money pumping doesn’t work is because Omega is unable to choose its policy based on its prediction of your second decision: If it could, you would want to switch back to b, because if you chose a, Omega would know that and you’d get 0 payoff. This makes the situation after the coinflip different from the original problem where Omega is able to see your decision and make its decision based on that.
In the Allais problem as stated, there’s no particular reason why the situation where you get to choose between $24,000, or $27,000 with 33⁄34 chance, differs depending on whether someone just offered it to you, or if they offered it to you only after you got <=34 on a d100.
Well, Omega doesn’t know which way the coin landed, but it does know that my policy is to choose a if the coin landed heads and b if the coin landed tails. I agree that the situation is different, because Omega’s state of knowledge is different, and that stops money pumping.
It’s just interesting that breaking the independence axiom does not lead to money pumping in this case. What if it doesn’t lead to money pumping in other cases too?
It seems that the axiom of independence doesn’t always hold for instrumental goals when you are playing a game.
Suppose you are playing a zero-sum game against Omega who can predict your move—either it has read your source code, or played enough games with you to predict you, including any pseudorandom number generator you have. You can make moves a or b, Omega can make moves c or d, and your payoff matrix is:
c d
a 0 4
b 4 1
U(a) = 0, U(b) = 1.
Now suppose we got a fair coin that Omega cannot predict, and can add a 0.5 probability of b to each:
U(0.5 a + 0.5 b) = min(0.5*0 + 0.5*4, 0.5*4 + 0.5*1) = 2
U(0.5 b + 0.5 b) = U(b) = 1
The preferences are reversed. However, the money-pumping doesn’t work:
I have my policy switch at b. You offer me to throw a fair coin and switch it to a if the coin comes up heads, for a cost of 0.1 utility. I say yes. You throw a coin, it comes up heads, you switch to a and offer me to switch it back to b for 0.1 utility. I say no thanks.
You could say that the mistake here was measuring utilities of policies. Outcomes have utility and policies only have expected utility. VNM axioms need not hold for policies. But money is not a terminal goal! Getting money is just a policy in the game of competing for scarce resources.
I wonder if there is a way to tell if someone’s preferences over policies is irrational, without knowing the game or outcomes.
In this case the only reason the money pumping doesn’t work is because Omega is unable to choose its policy based on its prediction of your second decision: If it could, you would want to switch back to b, because if you chose a, Omega would know that and you’d get 0 payoff. This makes the situation after the coinflip different from the original problem where Omega is able to see your decision and make its decision based on that.
In the Allais problem as stated, there’s no particular reason why the situation where you get to choose between $24,000, or $27,000 with 33⁄34 chance, differs depending on whether someone just offered it to you, or if they offered it to you only after you got <=34 on a d100.
Well, Omega doesn’t know which way the coin landed, but it does know that my policy is to choose a if the coin landed heads and b if the coin landed tails. I agree that the situation is different, because Omega’s state of knowledge is different, and that stops money pumping.
It’s just interesting that breaking the independence axiom does not lead to money pumping in this case. What if it doesn’t lead to money pumping in other cases too?