OK, time for further detail on the problem with pre-emptively submissive gnomes. Let’s focus on the case of total utilitarianism, and begin by looking at the decision in unlinked form, i.e. we assume that the gnome’s advice affects only one human if there is one in the room, and zero humans otherwise. Conditional on there being a human in cell B, the expected utility of the human in cell B buying a ticket for $x is, indeed, (1/3)(-x) + (2/3)(1-x) = 2⁄3 - x, so the breakeven is obviously at x = 2⁄3. However, if we also assume that the gnome in the other cell will give the same advice, we get (1/3)(-x) + 2(2/3)(1-x) = 4⁄3 - (5/3)x, with breakeven at x=4/5. In actual fact, the gnome’s reasoning, and the 4⁄5 answer, is correct. If tickets were being offered at a price of, say, 75 cents, then the overall outcome (conditional on there being a human in cell B) is indeed better if the humans buy at 75 cents than if they refuse to buy at 75 cents, because 3⁄4 is less than 4⁄5.
As I mentioned previously, in the case where the gnome only cares about total $ if there is a human in its cell, then 4⁄5 is correct before conditioning on the presence of a human, and it’s also correct after conditioning on the presence of a human; the number is 4⁄5 regardless. However, the situation we’re examining here is different, because the gnome cares about total $ even if no human is present. Thus we have a dilemma, because it appears that UDT is correct in advising the gnome to precommit to 2⁄3, but the above argument also suggests that after seeing a human in its cell it is correct for the gnome to advise 4⁄5.
The key distinction, analogously to mwenger’s answer to Psy-Kosh’s non-anthropic problem, has to do with the possibility of a gnome in an empty cell. For a total utilitarian gnome in an empty cell, any money at all spent in the other cell translates directly into negative utility. That gnome would prefer the human in the other cell to spend $0 at most, but of course there is no way to make this happen, since the other gnome has no way of knowing that this is the case.
The resolution to this problem is that, for linked decisions, you must (as UDT does) necessarily consider the effects of that decision over all a priori possible worlds affected by that decision. As it happens, this is the same thing as what you would do if you had the opportunity to precommit in advance.
It’s a bit trickier to justify why this should be the case, but the best argument I can come up with is to apply that same “linked decision” reasoning at one meta-level up, the level of “linked decision theories”. In short, by adopting a decision theory that ignores linked decisions in a priori possible worlds that are excluded by your observations, you are licensing yourself and other agents to do the same thing in future decisions, which you don’t want. If other agents follow this reasoning, they will give the “yea” answer in Psy-Kosh’s non-anthropic problem, but you don’t want them to do that.
Note that most of the time, decisions in worlds excluded by your observations do not usually tend to be “linked”. This is because exclusion by observation would usually imply that you receive a different observation in the other possible world, thus allowing you to condition your decision on that observation, and thereby unlinking the decisions. However, some rare problems like the Counterfactual Mugging and Psy-Kosh’s non-anthropic problem violate this tendency, and should therefore be treated differently.
Overall, then, the “linked decision theory” argument supports adopting UDT, and it means that you should consider all linked decisions in all a priori possible worlds.
Thanks a lot for your comments, they were very insightful for me. Let me play the Advocatus Diaboli here and argue from the perspective of a selfish agent against your reasoning (and thus also against my own, less refined version of it).
“I object to the identification ‘S = $B’. I do not care about the money owned by the person in cell B, I only do so if that person is me. I do not know whether the coin has come up heads or tails, but I do not care about how much money the other person that may have been in cell B had the coin come up differently would have paid or won. I only care about the money owned by the person in cell B in “this world”, where that person is me. I reject identifying myself with the other person that may have been in cell B had the coin come up differently, solely because that person would exist in the same cell as I do. My utility function thus cannot be expressed as a linear combination of $B and $C.
I would pay a counterfactual mugger. In that case, there is a transfer, as it were, between two possible selfes of mine that increases “our” total fortune. We are both both possible descendants of the same past-self, to which each of us is connected identically. The situation is quite different in the incubator case. There is no connection over a mutual past self between me and the other person that may have existed in cell B after a different outcome of the coin flip. This connection between past and future selves of mine is exactly what specifies my selfish goals. Actually, I don’t feel like the person that may have existed in cell B after a different outcome of the coin flip is “me” any more than the person in cell C is “me” (if that person exists). Since I will pay and win as much as the person in cell C (if they exist), I cannot win any money from them, and I don’t care about whether they exist at all, I think I should decide as an average utilitarian would. I will not pay more than $0.50.”
Is the egoist arguing this way mistaken? Or is our everyday notion of selfishness just not uniquely defined when it comes to the possibility of subjectively indistinguishable agents living in different “worlds”, since it rests on the dubious concept of personal identity? Can one understand selfishness both as caring about everyone living in subjectively identical circumstances as oneself (and their future selves), and as caring about everyone to whom one is directly connected only? Do these two possibilities correspond to SIA-egoists and SSA-egoists, respectively, which are both coherent possibilities?
First of all, I think your argument from connection of past/future selves is just a specific case of the more general argument for reflective consistency, and thus does not imply any kind of “selfishness” in and of itself. More detail is needed to specify a notion of selfishness.
I understand your argument against identifying yourself with another person who might counterfactually have been in the same cell, but the problem here is that if you don’t know how the coin actually came up you still have to assign amounts of “care” to the possible selves that you could actually be.
Let’s say that, as in my reasoning above, there are two cells, B and C; when the coin comes up tails humans are created in both cell B and cell C, but when the coin comes up heads a human is created in either cell B or cell C, with equal probability. Thus there are 3 “possible worlds”: 1) p=1/2 human in both cells 2) p=1/4 human in cell B, cell C empty 3) p=1/4 human in cell C, cell B empty
If you’re a selfish human and you know you’re in cell B, then you don’t care about world (3) at all, because there is no “you” in it. However, you still don’t know whether you’re in world (1) or (2), so you still have to “care” about both worlds. Moreover, in either world the “you” you care about is clearly the person in cell B, and so I think the only utility function that makes sense is S = $B. If you want to think about it in terms of either SSA-like or SIA-like assumptions, you get the same answer because both in world (1) and world (2) there is only a single observer who could be identified as “you”.
Now, what if you didn’t know whether you were in cell B or cell C? That’s where things are a little different. In that case, there are two observers in world (1), either of whom could be “you”. There are basically two different ways of assigning utility over the two different “yous” in world (1)---adding them together, like a total utilitarian, and averaging them, like an average utilitarian; the resulting values are x=2/3 and x=1/2 respectively. Moreover, the first approach is equivalent to SIA, and the second is equivalent to SSA.
However, the SSA answer has a property that none of the others do. If the gnome was to tell the human “you’re in cell B”, an SSA-using human would change their cutoff point from 1⁄2 to 2⁄3. This seems to be rather strange indeed, because whether the human is in cell B or in cell C is not in any way relevant to the payoff. No human with any of the other utility functions we’ve considered would change his/her answer upon being told that they are in cell B.
time for further detail on the problem with pre-emptively submissive gnomes.
One of the aspects of what makes LW what it is—people with serious expressions on their faces discuss the problems with pre-emptively submissive gnomes and nobody blinks an eye X-D
I guess your comment means that you must have blinked an eye, so your comment can’t be completely true. That said, as discussions of pre-emptively submissive gnomes go, I would generally expect the amount of eye-blinking on LW to be well below average ^_~
OK, time for further detail on the problem with pre-emptively submissive gnomes. Let’s focus on the case of total utilitarianism, and begin by looking at the decision in unlinked form, i.e. we assume that the gnome’s advice affects only one human if there is one in the room, and zero humans otherwise. Conditional on there being a human in cell B, the expected utility of the human in cell B buying a ticket for $x is, indeed, (1/3)(-x) + (2/3)(1-x) = 2⁄3 - x, so the breakeven is obviously at x = 2⁄3. However, if we also assume that the gnome in the other cell will give the same advice, we get (1/3)(-x) + 2(2/3)(1-x) = 4⁄3 - (5/3)x, with breakeven at x=4/5. In actual fact, the gnome’s reasoning, and the 4⁄5 answer, is correct. If tickets were being offered at a price of, say, 75 cents, then the overall outcome (conditional on there being a human in cell B) is indeed better if the humans buy at 75 cents than if they refuse to buy at 75 cents, because 3⁄4 is less than 4⁄5.
As I mentioned previously, in the case where the gnome only cares about total $ if there is a human in its cell, then 4⁄5 is correct before conditioning on the presence of a human, and it’s also correct after conditioning on the presence of a human; the number is 4⁄5 regardless. However, the situation we’re examining here is different, because the gnome cares about total $ even if no human is present. Thus we have a dilemma, because it appears that UDT is correct in advising the gnome to precommit to 2⁄3, but the above argument also suggests that after seeing a human in its cell it is correct for the gnome to advise 4⁄5.
The key distinction, analogously to mwenger’s answer to Psy-Kosh’s non-anthropic problem, has to do with the possibility of a gnome in an empty cell. For a total utilitarian gnome in an empty cell, any money at all spent in the other cell translates directly into negative utility. That gnome would prefer the human in the other cell to spend $0 at most, but of course there is no way to make this happen, since the other gnome has no way of knowing that this is the case.
The resolution to this problem is that, for linked decisions, you must (as UDT does) necessarily consider the effects of that decision over all a priori possible worlds affected by that decision. As it happens, this is the same thing as what you would do if you had the opportunity to precommit in advance.
It’s a bit trickier to justify why this should be the case, but the best argument I can come up with is to apply that same “linked decision” reasoning at one meta-level up, the level of “linked decision theories”. In short, by adopting a decision theory that ignores linked decisions in a priori possible worlds that are excluded by your observations, you are licensing yourself and other agents to do the same thing in future decisions, which you don’t want. If other agents follow this reasoning, they will give the “yea” answer in Psy-Kosh’s non-anthropic problem, but you don’t want them to do that.
Note that most of the time, decisions in worlds excluded by your observations do not usually tend to be “linked”. This is because exclusion by observation would usually imply that you receive a different observation in the other possible world, thus allowing you to condition your decision on that observation, and thereby unlinking the decisions. However, some rare problems like the Counterfactual Mugging and Psy-Kosh’s non-anthropic problem violate this tendency, and should therefore be treated differently.
Overall, then, the “linked decision theory” argument supports adopting UDT, and it means that you should consider all linked decisions in all a priori possible worlds.
Thanks a lot for your comments, they were very insightful for me. Let me play the Advocatus Diaboli here and argue from the perspective of a selfish agent against your reasoning (and thus also against my own, less refined version of it).
“I object to the identification ‘S = $B’. I do not care about the money owned by the person in cell B, I only do so if that person is me. I do not know whether the coin has come up heads or tails, but I do not care about how much money the other person that may have been in cell B had the coin come up differently would have paid or won. I only care about the money owned by the person in cell B in “this world”, where that person is me. I reject identifying myself with the other person that may have been in cell B had the coin come up differently, solely because that person would exist in the same cell as I do. My utility function thus cannot be expressed as a linear combination of $B and $C.
I would pay a counterfactual mugger. In that case, there is a transfer, as it were, between two possible selfes of mine that increases “our” total fortune. We are both both possible descendants of the same past-self, to which each of us is connected identically. The situation is quite different in the incubator case. There is no connection over a mutual past self between me and the other person that may have existed in cell B after a different outcome of the coin flip. This connection between past and future selves of mine is exactly what specifies my selfish goals. Actually, I don’t feel like the person that may have existed in cell B after a different outcome of the coin flip is “me” any more than the person in cell C is “me” (if that person exists). Since I will pay and win as much as the person in cell C (if they exist), I cannot win any money from them, and I don’t care about whether they exist at all, I think I should decide as an average utilitarian would. I will not pay more than $0.50.”
Is the egoist arguing this way mistaken? Or is our everyday notion of selfishness just not uniquely defined when it comes to the possibility of subjectively indistinguishable agents living in different “worlds”, since it rests on the dubious concept of personal identity? Can one understand selfishness both as caring about everyone living in subjectively identical circumstances as oneself (and their future selves), and as caring about everyone to whom one is directly connected only? Do these two possibilities correspond to SIA-egoists and SSA-egoists, respectively, which are both coherent possibilities?
First of all, I think your argument from connection of past/future selves is just a specific case of the more general argument for reflective consistency, and thus does not imply any kind of “selfishness” in and of itself. More detail is needed to specify a notion of selfishness.
I understand your argument against identifying yourself with another person who might counterfactually have been in the same cell, but the problem here is that if you don’t know how the coin actually came up you still have to assign amounts of “care” to the possible selves that you could actually be.
Let’s say that, as in my reasoning above, there are two cells, B and C; when the coin comes up tails humans are created in both cell B and cell C, but when the coin comes up heads a human is created in either cell B or cell C, with equal probability. Thus there are 3 “possible worlds”:
1) p=1/2 human in both cells
2) p=1/4 human in cell B, cell C empty
3) p=1/4 human in cell C, cell B empty
If you’re a selfish human and you know you’re in cell B, then you don’t care about world (3) at all, because there is no “you” in it. However, you still don’t know whether you’re in world (1) or (2), so you still have to “care” about both worlds. Moreover, in either world the “you” you care about is clearly the person in cell B, and so I think the only utility function that makes sense is S = $B. If you want to think about it in terms of either SSA-like or SIA-like assumptions, you get the same answer because both in world (1) and world (2) there is only a single observer who could be identified as “you”.
Now, what if you didn’t know whether you were in cell B or cell C? That’s where things are a little different. In that case, there are two observers in world (1), either of whom could be “you”. There are basically two different ways of assigning utility over the two different “yous” in world (1)---adding them together, like a total utilitarian, and averaging them, like an average utilitarian; the resulting values are x=2/3 and x=1/2 respectively. Moreover, the first approach is equivalent to SIA, and the second is equivalent to SSA.
However, the SSA answer has a property that none of the others do. If the gnome was to tell the human “you’re in cell B”, an SSA-using human would change their cutoff point from 1⁄2 to 2⁄3. This seems to be rather strange indeed, because whether the human is in cell B or in cell C is not in any way relevant to the payoff. No human with any of the other utility functions we’ve considered would change his/her answer upon being told that they are in cell B.
One of the aspects of what makes LW what it is—people with serious expressions on their faces discuss the problems with pre-emptively submissive gnomes and nobody blinks an eye X-D
I guess your comment means that you must have blinked an eye, so your comment can’t be completely true. That said, as discussions of pre-emptively submissive gnomes go, I would generally expect the amount of eye-blinking on LW to be well below average ^_~
I arched my eyebrow :-P