This is a very old post, but I have to say I don’t even understand the Axiom of Independence as presented here. It is stated:
The Axiom of Independence says that for any A, B, C, and p, you prefer A to B if and only if you prefer p A + (1-p) C to p B + (1-p) C.
If p A + (1-p) C and p B + (1-p) C, this means that both A and B are true if and only if C is false (two probabilities sum to 1 if and only if they are mutually exclusive and exhaustive). Which means A is true if and only if B is true, i.e. P(A↔B)=1. Since A and B have the same truth value with certainty, they are equivalent events. Like being a unmarried man is equivalent to being a bachelor. If we have to treat A and B as the same event, this would strongly suggest we have to assign them the same utility. But U(A)=U(B) is the same as A∼B, i.e. being indifferent between A and B. Which contradicts the assumption that A is preferred to B. Which would make the axiom, at least as stated here, necessarily false, not just false in some more exotic case involving indexical events.
This is also the point where I get stuck understanding the rest of the post. I assume I’m missing something basic here.
Maybe the axiom is intended as something like this:
U(A)>U(B) if and only if P(A|A∨C)U(A)+P(C|A∨C)U(C)>P(B|B∨C)U(B)+P(C|B∨C)U(C),
where
P(A|A∨C)=P(B|B∨C)=p and P(C|A∨C)=P(C|B∨C)=1−p.
Which could be interpreted as saying we should prefer A to B if and only if we would prefer “gambling between” (i.e. being uncertain about) A and C to gambling between B and C, assuming the relative odds are the same in both gambles.
But even in that case, it is unclear how the above can be interpreted like this:
This makes sense if p is a probability about the state of the world. (In the following, I’ll use “state” and “possible world” interchangeably.) In that case, what it’s saying is that what you prefer (e.g., A to B) in one possible world shouldn’t be affected by what occurs (C) in other possible worlds.
Unfortunately the last example with the vaccine also doesn’t help. It presents a case with just three options: I press A, I press B, I press C. It is unclear (to me) how preferring the third option (and being indifferent between the first two?) would contradict the independence axiom.
Even if we interpret the third option as a gamble between the first two, I’m not sure how this would be incompatible with Independence. But this interpretation doesn’t work anyway, since “I receive either vaccine A or B, but the opposite of what the other guy receives” isn’t equivalent to “I receive either vaccine A or B”.
I probably should have said this explicitly, but this post assumes prior understanding of Von Neumann–Morgenstern utility theorem and my description of the Axiom of Independence was only meant to remind someone of what the axiom is in case they forgot it, not meant to teach it to someone who never learned it. There’s a post on LW that tries to explain the theorem and the various axioms it assumes, or you can try to find another resource to learn it from.
Thanks. My second interpretation of the independence axiom seemed to be on track. The car example in the post you linked is formally analogous to your vaccine example. The mother is indifferent between giving the car to her son (A) or daughter (B) but prefers to throw a coin (C, such that C=0.5A+0.5B) to decide who gets it. Constructing it like this, according to the post, would contradict Independence. But the author argues that throwing the coin is not quite the same as simply 0.5A+0.5B, so independence isn’t violated.
This is similar to what I wrote, at the end, about your example above:
“I receive either vaccine A or B, but the opposite of what the other guy receives” isn’t equivalent to “I receive either vaccine A or B”.
Which would mean the example is compatible with the independence axiom.
Maybe there is a different example which would show that rational indexical preferences may contradict Independence, but I struggle to think of one.
“I receive either vaccine A or B, but the opposite of what the other guy receives” isn’t equivalent to “I receive either vaccine A or B”.
Yeah, I think this makes sense, at least if we assume a decision theory like EDT, where pressing C gives me a lot of evidence that the other guy also presses C, so I can think of the consequences of pressing C as “50% chance I receive A and the other guy receives B, 50% chance I receive B and the other guy receives A” which is not a gamble between the consequences of pressing A (we both receive A) and the consequences of pressing B (we both receive B) so Independence isn’t violated.
I think at the time I wrote the post, I was uncritically assuming CDT (under the impression that it was the mainstream academic decision theory), and under CDT you’re supposed to reason only about the causal consequences of your own decision, not how it correlates with the other guy’s decision. In that case the consequences of pressing C would be “50% chance I receive A, 50% chance I receive B” and then strictly preferring it would violate Independence. (Unless I say that pressing C also has the consequence of C having been pressed, and I prefer that over A or B having been pressed, but that seems like too much of a hack to me, or violates the spirit/purpose of decision theory.)
I hope this is correct and makes sense. (It’s been so long since I wrote this post that I had to try to guess/infer why I wrote what I wrote.) If so, I think that part of the post (about “preferential interaction”) is more of an argument against CDT than against Independence, but the other part (about physical interactions) still works?
Hm, interesting point about causal decision theory. It seems to me even with CDT I should expect as (causal) consequence of pressing C a higher probability that we get different vaccines than if I had only randomized between button A and B. Because I can expect some probability that the other guy also presses C (which then means we both do). Which would at least increase the overall probability that we get different vaccines, even if I’m not certain that we both press C. Though I find this confusing to reason about.
But anyway, this discussion of indexicals got me thinking of how to precisely express “actions” and “consequences” (outcomes?) in decision theory. And it seems that they should always trivially include an explicit or implicit indexical, not just in cases like the example above. Like for an action X, “I make X true”, and for an outcome Y, “I’m in a world where Y is true”. Something like that. Not sure how significant this is and whether there are counterexamples.
Yeah, it’s confusing to me too. Not sure how to think about this under CDT.
But anyway, this discussion of indexicals got me thinking of how to precisely express “actions” and “consequences” (outcomes?) in decision theory. And it seems that they should always trivially include an explicit or implicit indexical, not just in cases like the example above. Like for an action X, “I make X true”, and for an outcome Y, “I’m in a world where Y is true”. Something like that. Not sure how significant this is and whether there are counterexamples.
I actually got rid of all indexicals in UDT, because I found them too hard to think about, which seemed great for a while, until it occurred to me that humans plausibly have indexical values and maybe it’s not straightforward to translate them into non-indexical values.
See also this comment where I talk about how UDT expresses actions and consequences. Note that “program-that-is-you” is not an indexical, it’s a string that encodes your actual source code. This also makes UDT hard/impossible for humans to use, since we don’t have access to our literal source code. See also UDT shows that decision theory is more puzzling than ever which talks about these problems and others.
This is a very old post, but I have to say I don’t even understand the Axiom of Independence as presented here. It is stated:
If p A + (1-p) C and p B + (1-p) C, this means that both A and B are true if and only if C is false (two probabilities sum to 1 if and only if they are mutually exclusive and exhaustive). Which means A is true if and only if B is true, i.e. P(A↔B)=1. Since A and B have the same truth value with certainty, they are equivalent events. Like being a unmarried man is equivalent to being a bachelor. If we have to treat A and B as the same event, this would strongly suggest we have to assign them the same utility. But U(A)=U(B) is the same as A∼B, i.e. being indifferent between A and B. Which contradicts the assumption that A is preferred to B. Which would make the axiom, at least as stated here, necessarily false, not just false in some more exotic case involving indexical events.
This is also the point where I get stuck understanding the rest of the post. I assume I’m missing something basic here.
Maybe the axiom is intended as something like this:
U(A)>U(B) if and only if P(A|A∨C)U(A)+P(C|A∨C)U(C)>P(B|B∨C)U(B)+P(C|B∨C)U(C),
where
P(A|A∨C)=P(B|B∨C)=p and P(C|A∨C)=P(C|B∨C)=1−p.
Which could be interpreted as saying we should prefer A to B if and only if we would prefer “gambling between” (i.e. being uncertain about) A and C to gambling between B and C, assuming the relative odds are the same in both gambles.
But even in that case, it is unclear how the above can be interpreted like this:
Unfortunately the last example with the vaccine also doesn’t help. It presents a case with just three options: I press A, I press B, I press C. It is unclear (to me) how preferring the third option (and being indifferent between the first two?) would contradict the independence axiom.
Even if we interpret the third option as a gamble between the first two, I’m not sure how this would be incompatible with Independence. But this interpretation doesn’t work anyway, since “I receive either vaccine A or B, but the opposite of what the other guy receives” isn’t equivalent to “I receive either vaccine A or B”.
I probably should have said this explicitly, but this post assumes prior understanding of Von Neumann–Morgenstern utility theorem and my description of the Axiom of Independence was only meant to remind someone of what the axiom is in case they forgot it, not meant to teach it to someone who never learned it. There’s a post on LW that tries to explain the theorem and the various axioms it assumes, or you can try to find another resource to learn it from.
Thanks. My second interpretation of the independence axiom seemed to be on track. The car example in the post you linked is formally analogous to your vaccine example. The mother is indifferent between giving the car to her son (A) or daughter (B) but prefers to throw a coin (C, such that C=0.5A+0.5B) to decide who gets it. Constructing it like this, according to the post, would contradict Independence. But the author argues that throwing the coin is not quite the same as simply 0.5A+0.5B, so independence isn’t violated.
This is similar to what I wrote, at the end, about your example above:
Which would mean the example is compatible with the independence axiom.
Maybe there is a different example which would show that rational indexical preferences may contradict Independence, but I struggle to think of one.
Yeah, I think this makes sense, at least if we assume a decision theory like EDT, where pressing C gives me a lot of evidence that the other guy also presses C, so I can think of the consequences of pressing C as “50% chance I receive A and the other guy receives B, 50% chance I receive B and the other guy receives A” which is not a gamble between the consequences of pressing A (we both receive A) and the consequences of pressing B (we both receive B) so Independence isn’t violated.
I think at the time I wrote the post, I was uncritically assuming CDT (under the impression that it was the mainstream academic decision theory), and under CDT you’re supposed to reason only about the causal consequences of your own decision, not how it correlates with the other guy’s decision. In that case the consequences of pressing C would be “50% chance I receive A, 50% chance I receive B” and then strictly preferring it would violate Independence. (Unless I say that pressing C also has the consequence of C having been pressed, and I prefer that over A or B having been pressed, but that seems like too much of a hack to me, or violates the spirit/purpose of decision theory.)
I hope this is correct and makes sense. (It’s been so long since I wrote this post that I had to try to guess/infer why I wrote what I wrote.) If so, I think that part of the post (about “preferential interaction”) is more of an argument against CDT than against Independence, but the other part (about physical interactions) still works?
Hm, interesting point about causal decision theory. It seems to me even with CDT I should expect as (causal) consequence of pressing C a higher probability that we get different vaccines than if I had only randomized between button A and B. Because I can expect some probability that the other guy also presses C (which then means we both do). Which would at least increase the overall probability that we get different vaccines, even if I’m not certain that we both press C. Though I find this confusing to reason about.
But anyway, this discussion of indexicals got me thinking of how to precisely express “actions” and “consequences” (outcomes?) in decision theory. And it seems that they should always trivially include an explicit or implicit indexical, not just in cases like the example above. Like for an action X, “I make X true”, and for an outcome Y, “I’m in a world where Y is true”. Something like that. Not sure how significant this is and whether there are counterexamples.
Yeah, it’s confusing to me too. Not sure how to think about this under CDT.
I actually got rid of all indexicals in UDT, because I found them too hard to think about, which seemed great for a while, until it occurred to me that humans plausibly have indexical values and maybe it’s not straightforward to translate them into non-indexical values.
See also this comment where I talk about how UDT expresses actions and consequences. Note that “program-that-is-you” is not an indexical, it’s a string that encodes your actual source code. This also makes UDT hard/impossible for humans to use, since we don’t have access to our literal source code. See also UDT shows that decision theory is more puzzling than ever which talks about these problems and others.