As an aside, I’ll note that a lot of the solutions bandied around here to decision theory problems remind me of something from Magic: The Gathering which I took notice of back when I still followed it.
When I watched my friends play, one would frequently respond to another’s play with “Before you do that, I-” and use some card or ability to counter their opponent’s move. The rules of MTG let you do that sort of thing, but I always thought it was pretty silly, because they did not, in fact, have any idea that it would make sense to make that play until after seeing their opponent’s move. Once they see their opponent’s play, they get to retroactively decide what to do “before” their opponent can do it.
In real life, we don’t have that sort of privilege. If you’re in a Counterfactual Mugging scenario, for instance, you might be inclined to say “I ought to be the sort of person who would pay Omega, because if the coin had come up the other way, I would be making a lot of money now, so being that sort of person would have positive expected utility for this scenario.” But this is “Before you do that-” type reasoning. You could just as easily have ended up in a situation where Omega comes and tells you “I decided that if you were the sort of person who would not pay up in a Counterfactual Mugging scenario, I would give you a million dollars, but I’ve predicted that you would, so you get nothing.”
When you come up with a solution to an Omega-type problem involving some type of precommitment, it’s worth asking “would this precommitment have made sense when I was in a position of not knowing Omega existed, or having any idea what it would do even if it did exist?”
In real life, we sometimes have to make decisions dealing with agents who have some degree of predictive power with respect to our thought processes, but their motivations are generally not as arbitrary as those attributed to Omega in most hypotheticals.
Can you give a specific example of a bandied-around solution to a decision-theory problem where predictive power is necessary in order to implement that solution?
I suspect I disagree with you here—or, rather, I agree with the general principle you’ve articulated, but I suspect I disagree that it’s especially relevant to anything local—but it’s difficult to be sure without specifics.
With respect to the Counterfactual Mugging you reference in passing, for example, it seems enough to say “I ought to be the sort of person who would do whatever gets me positive expected utility”; I don’t have to specifically commit to pay or not pay. Isn’t it? But perhaps I’ve misunderstood the solution you’re rejecting.
Well, if your decision theory tells you you ought to be the sort of person who would pay up in a Counterfactual Mugging, because that gets you positive utility, then you could end up in with Omega coming and saying “I would have given you a million dollars if your decision theory said not to pay out in a counterfactual mugging, but since you would, you don’t get anything.”
When you know nothing about Omega, I don’t think there’s any positive expected utility in choosing to be the sort of person who would have positive expected utility in a Counterfactual Mugging scenario, because you have no reason to suspect it’s more likely than the inverted scenario where being that sort of person will get you negative utility. The probability distribution is flat, so the utilities cancel out.
Say Omega comes to you with a Counterfactual Mugging on Day 1. On Day 0, would you want to be the sort of person who pays out in a Counterfactual Mugging? No, because the probabilities of it being useful or harmful cancel out. On Day 1, when given the dilemma, do you want to be the sort of person who pays out in a Counterfactual Mugging? No, because now it only costs you money and you get nothing out of it.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Reasoning this way means, of course, that you don’t get the money in a situation where Omega would only pay you if it predicted you would pay up, but you do get the money in situations where Omega pays out only if you wouldn’t pay out. The latter possibility seems less salient from the “before you do that-” standpoint of a person contemplating a Counterfactual Mugging, but there’s no reason to assign it a lower probability before the fact. The best you can do is choose according to whatever has the highest expected utility at any given time.
Omega could also come and tell me “I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,” but I don’t let this influence my typing habits. You don’t want to alter your decision theories or general behavior in advance of specific events that are no more probable than their inversions.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Sure, I agree.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
Is this not true?
“I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,”
I agree that this is not something I can sensibly protect against. I’m not actually sure I would call it a decision theory problem at all.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
In the inversion I suggested to the Counterfactual Mugging, your payout is determined on the basis of whether you pay up in the Counterfactual Mugging. In the Counterfactual Mugging, Omega predicts whether you would pay out in the Counterfactual Mugging, and if you would, you get a 50% shot at a million dollars. In the inverted scenario, Omega predicts whether you would pay out in the Counterfactual Mugging scenario, and if you wouldn’t, you get a shot at a million dollars.
Being the sort of person who would pay out in a Counterfactual Mugging only brings positive expected utility if you expect the Counterfactual Mugging scenario to be more likely than the inverted Counterfactual Mugging scenario.
The inverted Counterfactual Mugging scenario, like the case where Omega rewards or punishes you based on your keyboard usage, isn’t exactly a decision theory problem, in that once it arises, you don’t get to make a decision, but it doesn’t need to be.
When the question is “should I be the sort of person who pays out in a Counterfactual Mugging?” if the chance of it being helpful is balanced out by an equal chance of it being harmful, then it doesn’t matter whether the situations that balance it out require you to make decisions at all, only that the expected utilities balance.
If you take as a premise “Omega simply doesn’t do that sort of thing, it only provides decision theory dilemmas where the results are dependent on how you would respond in this particular dilemma,” then our probability distribution is no longer flat, and being the sort of person who pays out in a Counterfactual Mugging scenario becomes utility maximizing. But this isn’t a premise we can take for granted. Omega is already posited as an entity which can judge your decision algorithms perfectly, and imposes dilemmas which are highly arbitrary.
As an aside, I’ll note that a lot of the solutions bandied around here to decision theory problems remind me of something from Magic: The Gathering which I took notice of back when I still followed it.
When I watched my friends play, one would frequently respond to another’s play with “Before you do that, I-” and use some card or ability to counter their opponent’s move. The rules of MTG let you do that sort of thing, but I always thought it was pretty silly, because they did not, in fact, have any idea that it would make sense to make that play until after seeing their opponent’s move. Once they see their opponent’s play, they get to retroactively decide what to do “before” their opponent can do it.
In real life, we don’t have that sort of privilege. If you’re in a Counterfactual Mugging scenario, for instance, you might be inclined to say “I ought to be the sort of person who would pay Omega, because if the coin had come up the other way, I would be making a lot of money now, so being that sort of person would have positive expected utility for this scenario.” But this is “Before you do that-” type reasoning. You could just as easily have ended up in a situation where Omega comes and tells you “I decided that if you were the sort of person who would not pay up in a Counterfactual Mugging scenario, I would give you a million dollars, but I’ve predicted that you would, so you get nothing.”
When you come up with a solution to an Omega-type problem involving some type of precommitment, it’s worth asking “would this precommitment have made sense when I was in a position of not knowing Omega existed, or having any idea what it would do even if it did exist?”
In real life, we sometimes have to make decisions dealing with agents who have some degree of predictive power with respect to our thought processes, but their motivations are generally not as arbitrary as those attributed to Omega in most hypotheticals.
Can you give a specific example of a bandied-around solution to a decision-theory problem where predictive power is necessary in order to implement that solution?
I suspect I disagree with you here—or, rather, I agree with the general principle you’ve articulated, but I suspect I disagree that it’s especially relevant to anything local—but it’s difficult to be sure without specifics.
With respect to the Counterfactual Mugging you reference in passing, for example, it seems enough to say “I ought to be the sort of person who would do whatever gets me positive expected utility”; I don’t have to specifically commit to pay or not pay. Isn’t it? But perhaps I’ve misunderstood the solution you’re rejecting.
Well, if your decision theory tells you you ought to be the sort of person who would pay up in a Counterfactual Mugging, because that gets you positive utility, then you could end up in with Omega coming and saying “I would have given you a million dollars if your decision theory said not to pay out in a counterfactual mugging, but since you would, you don’t get anything.”
When you know nothing about Omega, I don’t think there’s any positive expected utility in choosing to be the sort of person who would have positive expected utility in a Counterfactual Mugging scenario, because you have no reason to suspect it’s more likely than the inverted scenario where being that sort of person will get you negative utility. The probability distribution is flat, so the utilities cancel out.
Say Omega comes to you with a Counterfactual Mugging on Day 1. On Day 0, would you want to be the sort of person who pays out in a Counterfactual Mugging? No, because the probabilities of it being useful or harmful cancel out. On Day 1, when given the dilemma, do you want to be the sort of person who pays out in a Counterfactual Mugging? No, because now it only costs you money and you get nothing out of it.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Reasoning this way means, of course, that you don’t get the money in a situation where Omega would only pay you if it predicted you would pay up, but you do get the money in situations where Omega pays out only if you wouldn’t pay out. The latter possibility seems less salient from the “before you do that-” standpoint of a person contemplating a Counterfactual Mugging, but there’s no reason to assign it a lower probability before the fact. The best you can do is choose according to whatever has the highest expected utility at any given time.
Omega could also come and tell me “I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,” but I don’t let this influence my typing habits. You don’t want to alter your decision theories or general behavior in advance of specific events that are no more probable than their inversions.
Sure, I agree.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
Is this not true?
I agree that this is not something I can sensibly protect against. I’m not actually sure I would call it a decision theory problem at all.
In the inversion I suggested to the Counterfactual Mugging, your payout is determined on the basis of whether you pay up in the Counterfactual Mugging. In the Counterfactual Mugging, Omega predicts whether you would pay out in the Counterfactual Mugging, and if you would, you get a 50% shot at a million dollars. In the inverted scenario, Omega predicts whether you would pay out in the Counterfactual Mugging scenario, and if you wouldn’t, you get a shot at a million dollars.
Being the sort of person who would pay out in a Counterfactual Mugging only brings positive expected utility if you expect the Counterfactual Mugging scenario to be more likely than the inverted Counterfactual Mugging scenario.
The inverted Counterfactual Mugging scenario, like the case where Omega rewards or punishes you based on your keyboard usage, isn’t exactly a decision theory problem, in that once it arises, you don’t get to make a decision, but it doesn’t need to be.
When the question is “should I be the sort of person who pays out in a Counterfactual Mugging?” if the chance of it being helpful is balanced out by an equal chance of it being harmful, then it doesn’t matter whether the situations that balance it out require you to make decisions at all, only that the expected utilities balance.
If you take as a premise “Omega simply doesn’t do that sort of thing, it only provides decision theory dilemmas where the results are dependent on how you would respond in this particular dilemma,” then our probability distribution is no longer flat, and being the sort of person who pays out in a Counterfactual Mugging scenario becomes utility maximizing. But this isn’t a premise we can take for granted. Omega is already posited as an entity which can judge your decision algorithms perfectly, and imposes dilemmas which are highly arbitrary.