1) I argue that my answer should depend on Omega because I illustrate different ways that Omega could operate that would reasonably change what the correct decision was. The problem statement can’t specify that my answer shouldn’t depend on how Omega works. I guess it can specify that I don’t know anything about Omega other than that he’s right and I would have to make assumptions based on what seemed like the most plausible way for that to happen. Also many statements of the problem don’t specify that Omega is always right. Besides even if that were the case, how could I possibly know for sure that Omega was always right (or that I was actually talking to Omega).
2) Maybe its not the best assumption. But if Omega is running simulations you should have some expectation of the probability of you being a simulation and making it proportional to the number of simulations vs. real you’s seems reasonable.
3) Well when simulated-you is initiated is it psychologically contiguous with real-you. After the not much time that the simulation probably takes, it hasn’t diverged that much.
4) Why should it have a sharp discontinuity when Omega becomes an imperfect predictor? This would only happen if you had a sharp discontinuity about how much you cared about copies of yourself as they became less than perfect. Why should one discontinuity be bad but not the other?
OK. Perhaps this theory isn’t terribly general but it is a reasonably coherent theory that extends to some class of other examples and produces the “correct” answer in these ones.
Maybe its not the best assumption. But if Omega is running simulations you should have some expectation of the probability of you being a simulation and making it proportional to the number of simulations vs. real you’s seems reasonable.
I don’t think this is going to work.
Consider a variation of ‘counterfactual mugging’ where Omega asks for $101 if heads but only gives back $100 if (tails and) it predicts the player would have paid up. Suppose that, for whatever reason, Omega runs 2 simulations of you rather than just 1. Then by the logic above, you should believe you are a simulation with probability 2⁄3, and so because 2x100 > 1x101, you should pay up. This is ‘intuitively wrong’ because if you choose to play this game many times then Omega will just take more and more of your money.
In counterfactual mugging (the original version, that is), to be ‘reflectively consistent’, you need to pay up regardless of whether Omega’s simulation is in some obscure sense ‘subjectively continuous’ with you.
Let me characterize your approach to this problem as follows:
You classify each possible state of the game as either being ‘you’ or ‘not you’.
You decide what to do on the basis of the assumption that you are a sample of one drawn from a uniform distribution over the set of states classified as being ‘you’.
Note that this approach gets the answer wrong for the absent-minded driver problem unless you can somehow force yourself to believe that the copy of you at the second intersection is really a mindless robot whose probability of turning is (for whatever reason) guaranteed to be the same as yours.
Though perhaps you are right and I need to be similarly careful to avoid counting some outcomes more often than others, which might be a problem, for example, if Omega ran different numbers of simulations depending on the coin flip.
So if Omega simulates several copies of me, it can’t be that both of them, by themselves have the power to make Omega decide to give real-me the money. So I have to give Omega money twice in simulation to get real-me the money once.
As for the absent-minded driver problem the problem here is that the probabilistic approach overcounts probability in some situations but not in others. It’s like playing the following game:
Phase 1: I flip a coin
If it was heads, there is a Phase 1.5 in which you get to guess its value and then are given an amnesia pill
Phase 2: You get to guess whether the coin came up heads and if you are right you get $1.
Using the bad analysis from the absent-minded driver problem. Your strategy is to always guess that it is heads with probability p. Suppose that there is a probability alpha that you are in phase 1.5 when you guess, and 1-alpha that you are in phase 2.
Well your expected payoff is then
(alpha)(p) + (1-alpha)*(1/2)
This is clearly silly. This is because if the coin came up heads they counted your strategy twice. I guess to fix this in general, you need to pick a time at which you make your averaging over possible yous. (For example, only count it at phase 2).
For the absent-minded driver problem, you could either choose at the first intersection you come to, in which case you have
1*(p^2+4p(1-p)) + 0(p+4(1-p))
Or the last intersection you come to in which case alpha = 1-p and you have
(1-p)*0 + p(p + 4(1-p))
(the 0 because if X is your last intersection you get 0)
Both give the correct answer.
Phase 1: I flip a coin If it was heads, there is a Phase 1.5 in which you get to guess its value and then are given an amnesia pill Phase 2: You get to guess whether the coin came up heads and if you are right you get $1.
This is (a variant of) the Sleeping Beauty problem. I’m guessing you must be new here—this is an old ‘chestnut’ that we’ve done to death several times. :-)
(1-p)*0 + p(p + 4(1-p))
(the 0 because if X is your last intersection you get 0) Both give the correct answer.
Good stuff. But now here’s a really stupid idea for you:
Suppose you’re going to play Counterfactual Mugging with an Omega who (for argument’s sake) doesn’t create a conscious simulation of you. But your friend Bill has a policy that if you ever have to play counterfactual mugging, and the coin lands tails then he will create a simulation of you as you were just prior to the game and make your copy have an experience indistinguishable from the experience you would have had of Omega asking you for money (as though the coin had landed heads). Then following your approach, surely you ought now to pay up (whereas you wouldn’t have previously)? Despite the fact that your friend Bill is penniless, and his actions have no effect on Omega or your payoff in the real world?
I don’t see why you think I should pay if Bill is involved. Knowing Bill’s behavior, I think that there’s a 50% chance that I am real, any paying earns me -$1000, and there’s a 50% chance that I am a Bill-simulation and paying earns me $0. Hence paying earns me an expected -$500.
If you know there is going to be a simulation then your subjective probability for the state of the real coin is that it’s heads with probability 1⁄2. And if the coin is really tails then, assuming Omega is perfect, your action of ‘giving money’ (in the simulation) seems to be “determining” whether or not you receive money (in the real world).
(Perhaps you’ll simply take this as all the more reason to rule out the possibility that there can be a perfect Omega that doesn’t create a conscious simulation of you? Fair enough.)
I’m not sure I would buy this argument unless you could claim that my Bob-simulation’s actions would cause Omega to give or not give me money. At very least it should depend on how Omega makes his prediction.
Perhaps a clearer variation goes as follows: Bill arranges so that if the coin is tails then (a) he will temporarily receive your winnings, if you get any, and (b) he will do a flawless imitation of Omega asking for money.
If you pay Bill then he returns both what you paid and your winnings (which you’re guaranteed to have, by hypothesis). If you don’t pay him then he has no winnings to give you.
Well look: If the real coin is tails and you pay up, then (assuming Omega is perfect, but otherwise irrespectively of how it makes its prediction) you know with certainty that you get the prize. If you don’t pay up then you would know with certainty that you don’t get the prize. The absence of a ‘causal arrow’ pointing from your decision to pay to Omega’s decision to pay becomes irrelevant in light of this.
(One complication which I think is reasonable to consider here is ‘what if physics is indeterministic and so knowing your prior state doesn’t permit Omega (or Bill) to calculate with certainty what you will do?’ Here I would generalize the game slightly so that if Omega calculates that your probability of paying up is p then you receive proportion p of the prize. Then everything else goes through unchanged—Omega and Bill will now calculate the same probability that you pay up.)
OK. I am uncomfortable with the idea of dealing with the situation where Omega is actually perfect.
I guess this boils down to me being not quite convinced by the arguments for one-boxing in Newcomb’s problem without further specification of how Omega operates.
At first sight it appears to be isomorphic to Newcomb’s problem. However, a couple of extra details have been thrown in:
A person’s decisions are a product of both conscious deliberation and predetermined unconscious factors beyond their control.
“Omega” only has access to the latter.
Now, I agree that when you have an imperfect Omega, even though it may be very accurate, you can’t rule out the possibility that it can only “see” the unfree part of your will, in which case you should “try as hard as you can to two-box (but perhaps not succeed).” However, if Omega has even “partial access” to the “free part” of your will then it will usually be best to one-box.
I did not know about it, thanks for pointing it out. It’s Simpson’s paradox the decision theory problem.
On the other hand (ignoring issues of Omega using magic or time travel, or you making precommitments), isn’t Newcomb’s problem always like this in that there is no direct causal relationship between your decision and his prediction, just that they share some common causation.
1) Yes, perfection is terribly unrealistic, but I think it gets too complicated to be interesting if it’s done any other way. It’s like a limit in mathematics—in fact, it should be the limit of relating to any prediction process as that process approaches perfection, or else you have a nasty discontinuity in your decision process, because all perfect processes can just be defined as “it’s perfect.”
2) Okay.
3) Statistical correlation, but not causal, so my definition would still tell them apart. In short, if you could throw me into the sun and then simulate me to atom-scale perfection, I would not want you to. This is because continuity is important to my sense of self.
4) Because any solution to the problem of consciousness and relationship between how much like you it is and how much you identify with it is going to be arbitrary. And so the picture in my head is is that the function of how much you would be willing to pay becomes multivalued as Omega becomes imperfect. And my brain sees a multivalued function and returns “not actually a function. Do not use.”
1) OK taking a limit is an idea I hadn’t thought of. It might even defeat my argument that your answer depends on how Omega achieves this. On the other hand:
a) I am not sure what the rest of my beliefs would look like anymore if I saw enough evidence to convince me that Omega was right all the time with probability 1-1/3^^^3 .
b) I doubt that the above is even possible, since given my argument you shouldn’t be able to convince me that the probability is less than say 10^-10 that I am a simulation talking to something that is not actually Omega.
3) I am not sure why you think that the simulation is not causally a copy of you. Either that or I am not sure what your distinction between statistical and causal is.
3+4) I agree that one of the weaknesses of this theory is that it depends heavily, among other things, on a somewhat controversial theory of identity/ what it means to win. Though I don’t see why the amount that you identify with an imperfect copy of yourself should be arbitrary, or at very least if that’s the case why its a problem for the dependence of your actions on Omega’s degree of perfection to be arbitrary, but not a problem for your identification with imperfect copies of yourself to be.
1) I argue that my answer should depend on Omega because I illustrate different ways that Omega could operate that would reasonably change what the correct decision was. The problem statement can’t specify that my answer shouldn’t depend on how Omega works. I guess it can specify that I don’t know anything about Omega other than that he’s right and I would have to make assumptions based on what seemed like the most plausible way for that to happen. Also many statements of the problem don’t specify that Omega is always right. Besides even if that were the case, how could I possibly know for sure that Omega was always right (or that I was actually talking to Omega).
2) Maybe its not the best assumption. But if Omega is running simulations you should have some expectation of the probability of you being a simulation and making it proportional to the number of simulations vs. real you’s seems reasonable.
3) Well when simulated-you is initiated is it psychologically contiguous with real-you. After the not much time that the simulation probably takes, it hasn’t diverged that much.
4) Why should it have a sharp discontinuity when Omega becomes an imperfect predictor? This would only happen if you had a sharp discontinuity about how much you cared about copies of yourself as they became less than perfect. Why should one discontinuity be bad but not the other?
OK. Perhaps this theory isn’t terribly general but it is a reasonably coherent theory that extends to some class of other examples and produces the “correct” answer in these ones.
I don’t think this is going to work.
Consider a variation of ‘counterfactual mugging’ where Omega asks for $101 if heads but only gives back $100 if (tails and) it predicts the player would have paid up. Suppose that, for whatever reason, Omega runs 2 simulations of you rather than just 1. Then by the logic above, you should believe you are a simulation with probability 2⁄3, and so because 2x100 > 1x101, you should pay up. This is ‘intuitively wrong’ because if you choose to play this game many times then Omega will just take more and more of your money.
In counterfactual mugging (the original version, that is), to be ‘reflectively consistent’, you need to pay up regardless of whether Omega’s simulation is in some obscure sense ‘subjectively continuous’ with you.
Let me characterize your approach to this problem as follows:
You classify each possible state of the game as either being ‘you’ or ‘not you’.
You decide what to do on the basis of the assumption that you are a sample of one drawn from a uniform distribution over the set of states classified as being ‘you’.
Note that this approach gets the answer wrong for the absent-minded driver problem unless you can somehow force yourself to believe that the copy of you at the second intersection is really a mindless robot whose probability of turning is (for whatever reason) guaranteed to be the same as yours.
Though perhaps you are right and I need to be similarly careful to avoid counting some outcomes more often than others, which might be a problem, for example, if Omega ran different numbers of simulations depending on the coin flip.
So if Omega simulates several copies of me, it can’t be that both of them, by themselves have the power to make Omega decide to give real-me the money. So I have to give Omega money twice in simulation to get real-me the money once.
As for the absent-minded driver problem the problem here is that the probabilistic approach overcounts probability in some situations but not in others. It’s like playing the following game:
Phase 1: I flip a coin If it was heads, there is a Phase 1.5 in which you get to guess its value and then are given an amnesia pill Phase 2: You get to guess whether the coin came up heads and if you are right you get $1.
Using the bad analysis from the absent-minded driver problem. Your strategy is to always guess that it is heads with probability p. Suppose that there is a probability alpha that you are in phase 1.5 when you guess, and 1-alpha that you are in phase 2.
Well your expected payoff is then (alpha)(p) + (1-alpha)*(1/2)
This is clearly silly. This is because if the coin came up heads they counted your strategy twice. I guess to fix this in general, you need to pick a time at which you make your averaging over possible yous. (For example, only count it at phase 2).
For the absent-minded driver problem, you could either choose at the first intersection you come to, in which case you have
1*(p^2+4p(1-p)) + 0(p+4(1-p))
Or the last intersection you come to in which case alpha = 1-p and you have
(1-p)*0 + p(p + 4(1-p))
(the 0 because if X is your last intersection you get 0) Both give the correct answer.
This is (a variant of) the Sleeping Beauty problem. I’m guessing you must be new here—this is an old ‘chestnut’ that we’ve done to death several times. :-)
Good stuff. But now here’s a really stupid idea for you:
Suppose you’re going to play Counterfactual Mugging with an Omega who (for argument’s sake) doesn’t create a conscious simulation of you. But your friend Bill has a policy that if you ever have to play counterfactual mugging, and the coin lands tails then he will create a simulation of you as you were just prior to the game and make your copy have an experience indistinguishable from the experience you would have had of Omega asking you for money (as though the coin had landed heads). Then following your approach, surely you ought now to pay up (whereas you wouldn’t have previously)? Despite the fact that your friend Bill is penniless, and his actions have no effect on Omega or your payoff in the real world?
I don’t see why you think I should pay if Bill is involved. Knowing Bill’s behavior, I think that there’s a 50% chance that I am real, any paying earns me -$1000, and there’s a 50% chance that I am a Bill-simulation and paying earns me $0. Hence paying earns me an expected -$500.
If you know there is going to be a simulation then your subjective probability for the state of the real coin is that it’s heads with probability 1⁄2. And if the coin is really tails then, assuming Omega is perfect, your action of ‘giving money’ (in the simulation) seems to be “determining” whether or not you receive money (in the real world).
(Perhaps you’ll simply take this as all the more reason to rule out the possibility that there can be a perfect Omega that doesn’t create a conscious simulation of you? Fair enough.)
I’m not sure I would buy this argument unless you could claim that my Bob-simulation’s actions would cause Omega to give or not give me money. At very least it should depend on how Omega makes his prediction.
Perhaps a clearer variation goes as follows: Bill arranges so that if the coin is tails then (a) he will temporarily receive your winnings, if you get any, and (b) he will do a flawless imitation of Omega asking for money.
If you pay Bill then he returns both what you paid and your winnings (which you’re guaranteed to have, by hypothesis). If you don’t pay him then he has no winnings to give you.
Well look: If the real coin is tails and you pay up, then (assuming Omega is perfect, but otherwise irrespectively of how it makes its prediction) you know with certainty that you get the prize. If you don’t pay up then you would know with certainty that you don’t get the prize. The absence of a ‘causal arrow’ pointing from your decision to pay to Omega’s decision to pay becomes irrelevant in light of this.
(One complication which I think is reasonable to consider here is ‘what if physics is indeterministic and so knowing your prior state doesn’t permit Omega (or Bill) to calculate with certainty what you will do?’ Here I would generalize the game slightly so that if Omega calculates that your probability of paying up is p then you receive proportion p of the prize. Then everything else goes through unchanged—Omega and Bill will now calculate the same probability that you pay up.)
OK. I am uncomfortable with the idea of dealing with the situation where Omega is actually perfect.
I guess this boils down to me being not quite convinced by the arguments for one-boxing in Newcomb’s problem without further specification of how Omega operates.
Do you know about the “Smoking Lesion” problem?
At first sight it appears to be isomorphic to Newcomb’s problem. However, a couple of extra details have been thrown in:
A person’s decisions are a product of both conscious deliberation and predetermined unconscious factors beyond their control.
“Omega” only has access to the latter.
Now, I agree that when you have an imperfect Omega, even though it may be very accurate, you can’t rule out the possibility that it can only “see” the unfree part of your will, in which case you should “try as hard as you can to two-box (but perhaps not succeed).” However, if Omega has even “partial access” to the “free part” of your will then it will usually be best to one-box.
Or at least this is how I like to think about it.
I did not know about it, thanks for pointing it out. It’s Simpson’s paradox the decision theory problem.
On the other hand (ignoring issues of Omega using magic or time travel, or you making precommitments), isn’t Newcomb’s problem always like this in that there is no direct causal relationship between your decision and his prediction, just that they share some common causation.
1) Yes, perfection is terribly unrealistic, but I think it gets too complicated to be interesting if it’s done any other way. It’s like a limit in mathematics—in fact, it should be the limit of relating to any prediction process as that process approaches perfection, or else you have a nasty discontinuity in your decision process, because all perfect processes can just be defined as “it’s perfect.”
2) Okay.
3) Statistical correlation, but not causal, so my definition would still tell them apart. In short, if you could throw me into the sun and then simulate me to atom-scale perfection, I would not want you to. This is because continuity is important to my sense of self.
4) Because any solution to the problem of consciousness and relationship between how much like you it is and how much you identify with it is going to be arbitrary. And so the picture in my head is is that the function of how much you would be willing to pay becomes multivalued as Omega becomes imperfect. And my brain sees a multivalued function and returns “not actually a function. Do not use.”
1) OK taking a limit is an idea I hadn’t thought of. It might even defeat my argument that your answer depends on how Omega achieves this. On the other hand:
a) I am not sure what the rest of my beliefs would look like anymore if I saw enough evidence to convince me that Omega was right all the time with probability 1-1/3^^^3 .
b) I doubt that the above is even possible, since given my argument you shouldn’t be able to convince me that the probability is less than say 10^-10 that I am a simulation talking to something that is not actually Omega.
3) I am not sure why you think that the simulation is not causally a copy of you. Either that or I am not sure what your distinction between statistical and causal is.
3+4) I agree that one of the weaknesses of this theory is that it depends heavily, among other things, on a somewhat controversial theory of identity/ what it means to win. Though I don’t see why the amount that you identify with an imperfect copy of yourself should be arbitrary, or at very least if that’s the case why its a problem for the dependence of your actions on Omega’s degree of perfection to be arbitrary, but not a problem for your identification with imperfect copies of yourself to be.