“You go to visit your friend Ann, and her mom pulls you into the kitchen, where two boxes are sitting on a table. She tells you that box A has either $1 billion or $0, and box B has $1,000. She says you can take both boxes or just A, and that if she predicted you take box B she didn’t put anything in A. She has done this to 100 of Anne’s friends and has only been wrong for one of them. She is a great predictor because she has been spying on your philosophy class and reading your essays.”
To be properly isomorphic to the Newcomb’s problem, the chance of the predictor being wrong should approximate to zero.
If I thought that the chance of my friend’s mother being wrong approximated to zero, I would of course choose to one-box. If I expected her to be an imperfect predictor who assumed I would behave as if I were in the real Newcomb’s problem with a perfect predictor, then I would choose to two-box.
In Newcomb’s Problem, if you choose on the basis of which choice is consistent with a higher expected return, then you would choose to one-box. You know that your choice doesn’t cause the box to be filled, but given the knowledge that whether the money is in the box or not is contingent on a perfect predictor’s assessment of whether or not you were likely to one-box, you should assign different probabilities to the box containing the money depending on whether you one-box or two-box. Since your own mental disposition is evidence of whether the money is in the box or not, you can behave as if the contents were determined by your choice.
To be properly isomorphic to the Newcomb’s problem, the chance of the predictor being wrong should approximate to zero.
If I thought that the chance of my friend’s mother being wrong approximated to zero, I would of course choose to one-box. If I expected her to be an imperfect predictor who assumed I would behave as if I were in the real Newcomb’s problem with a perfect predictor, then I would choose to two-box.
Hm, I think I still don’t understand the one-box perspective, then. Are you saying that if the predictor is wrong with probability p, you would take two-boxes for high p and one box for a sufficiently small p (or just for p=0)? What changes as p shrinks?
Or what if Omega/Ann’s mom is a perfect predictor, but for a random 1% of the time decides to fill the boxes as if it made the opposite prediction, just to mess with you? If you one-box for p=0, you should believe that taking one box is correct (and generates $1 million more) in 99% of cases and that two boxes is correct (and generates $1000 more) in 1% of cases. So taking one box should still have a far higher expected value. But the perfect predictor who sometimes pretends to be wrong behaves exactly the same as an imperfect predictor who is wrong 1% of the time.
You choose the boxes according to the expected value of each box choice. For a 99% accurate predictor, the expected value of one-boxing is $990,000,000 (you get a billion 99% of the time, and nothing 1% of the time,) while the expected value of two-boxing is $10,001,000 (you get a thousand 99% of the time, and one billion and one thousand 1% of the time.)
The difference between this scenario and the one you posited before, where Ann’s mom makes her prediction by reading your philosophy essays, is that she’s presumably predicting on the basis of how she would expect you to choose if you were playing Omega. If you’re playing against an agent who you know will fill the boxes according to how you would choose if you were playing Omega (we’ll call it Omega-1,) then you should always two-box (if you would one-box against Omega, both boxes will contain money, so you get the contents of both. If you would two-box against Omega, only one box would contain money, and if you one-box you’ll get the empty one.)
An imperfect predictor with random error is a different proposition from an imperfect predictor with nonrandom error.
Of course, if I were dealing with this dilemma in real life, my choice would be heavily influenced by considerations such as how likely it is that Ann’s mom really has billions of dollars to give away.
The difference between this scenario and the one you posited before, where Ann’s mom makes her prediction by reading your philosophy essays, is that she’s presumably predicting on the basis of how she would expect you to choose if you were playing Omega.
Ok, but what if Ann’s mom is right 99% of the time about how you would choose when playing her?
I agree that one-boxers make more money, with the numbers you used, but I don’t think that those are the appropriate expected values to consider. Conditional on the fact that the boxes have already been filled, two-boxing has a $1000 higher expected value. If I know only one box is filled, I should take both. If I know both boxes are filled, I should take both. If I know I’m in one of those situations but not sure of which it is, I should still take both.
Another analogous situation would be that you walk into an exam, and the professor (who is a perfect or near-perfect predictor) announces that he has written down a list of people whom he has predicted will get fewer than half the questions right. If you are on that list, he will add 100 points to your score at the end. The people who get fewer than half of the questions right get higher scores, but you should still try to get questions right on the test… right? If not, does the answer change if the professor posts the list on the board?
I still think I’m missing something, since a lot of people have thought carefully about this and come to a different conclusion from me, but I’m still not sure what it is. :/
Conditional on the fact that the boxes have already been filled, two-boxing has a $1000 higher expected value. If I know only one box is filled, I should take both. If I know both boxes are filled, I should take both. If I know I’m in one of those situations but not sure of which it is, I should still take both.
You are focusing too much on the “already have been filled”, as if the particular time of your particular decision is relevant. But if your decision isn’t random (and yours isn’t), then any individual decision is dependent on the decision algorithm you follow—and can be calculated in exactly the same manner, regardless of time. Therefore in a sense your decision has been made BEFORE the filling of the boxes, and can affect their contents.
You may consider it easier to wrap your head around this if you think of the boxes being filled according to what result the decision theory you currently have would return in the situation, instead of what decision you’ll make in the future. That helps keep in mind that causality still travels only one direction, but that a good predictor simply knows the decision you’ll make before you make it and can act accordingly.
Ok, but what if Ann’s mom is right 99% of the time about how you would choose when playing her?
I would one-box. I gave the relevant numbers on this in my previous comment; one-boxing has an expected value of $990,000,000 to the expected $10,001,000 if you two-box.
I agree that one-boxers make more money, with the numbers you used, but I don’t think that those are the appropriate expected values to consider. Conditional on the fact that the boxes have already been filled, two-boxing has a $1000 higher expected value. If I know only one box is filled, I should take both. If I know both boxes are filled, I should take both. If I know I’m in one of those situations but not sure of which it is, I should still take both.
When you’re dealing with a problem involving an effective predictor of your own mental processes (it’s not necessary for such a predictor to be perfect for this reasoning to become salient, it just makes the problems simpler,) your expectation of what the predictor will do or already have done will be at least partly dependent on what you intend to do yourself. You know that either the opaque box is filled, or it is not, but the probability you assign to the box being filled depends on whether you intend to open it or not.
Let’s try a somewhat different scenario. Suppose I have a time machine that allows me to travel back a day in the past. Doing so creates a stable time loop, like the time turners in Harry Potter or HPMoR (on a side note, our current models of relativity suggest that such loops are possible, if very difficult to contrive.) You’re angry at me because I’ve insulted your hypothetical scenario, and are considering hitting me in retaliation. But you happen to know that I retaliate against people who hit me by going back in time and stealing from them, which I always get away with due to having perfect alibis (the police don’t believe in my time machine.) You do not know whether I’ve stolen from you or not, but if I have, it’s already happened. You would feel satisfied by hitting me, but it’s not worth being stolen from. Do you choose to hit me or not?
Another analogous situation would be that you walk into an exam, and the professor (who is a perfect or near-perfect predictor) announces that he has written down a list of people whom he has predicted will get fewer than half the questions right. If you are on that list, he will add 100 points to your score at the end. The people who get fewer than half of the questions right get higher scores, but you should still try to get questions right on the test… right? If not, does the answer change if the professor posts the list on the board?
If the professor is a perfect predictor, then I would deliberately get most of the problems wrong, thereby all but guaranteeing a score of over 100 points. I would have to be very confident that I would get a score below fifty even if I weren’t trying to on purpose before trying to get all the questions right would give me a higher expected score than trying to get most of the questions wrong.
If the professor posts the list on the board, then of course it should affect the answer. If my name isn’t on the list, then he’s not going to add the 100 points to my test in any case, so my only recourse to maximizing my grade is to try my best on the test. If my name is on the list, then he’s already predicted that I’m going to score below 50, so whether he’s a perfect predictor or not, I should try to do well so that he’s adding 100 points to as high a score as I can manage.
The difference between the scenario where he writes the names on the board and the scenario where he doesn’t is that in the former, my expectations of his actions don’t vary according to my own, whereas in the latter, they do.
If the professor posts the list on the board, then of course it should affect the answer. If my name isn’t on the list, then he’s not going to add the 100 points to my test in any case, so my only recourse to maximizing my grade is to try my best on the test. If my name is on the list, then he’s already predicted that I’m going to score below 50, so whether he’s a perfect predictor or not, I should try to do well so that he’s adding 100 points to as high a score as I can manage.
I believe you are making a mistake. Specifically, you are implementing a decision algorithm that ensures that “you lose” is a correct self fulfilling prophecy (in fact you ensure that it is the only valid prediction he could make). I would throw the test (score in the 40s) even when my name is not on the list.
The difference between the scenario where he writes the names on the board and the scenario where he doesn’t is that in the former, my expectations of his actions don’t vary according to my own, whereas in the latter, they do.
I believe you are making a mistake. Specifically, you are implementing a decision algorithm that ensures that “you lose” is a correct self fulfilling prophecy (in fact you ensure that it is the only valid prediction he could make). I would throw the test (score in the 40s) even when my name is not on the list.
If I were in a position to predict that this were the sort of thing the professor might do, then I would precommit to throwing the test should he implement such a procedure. But you could just as easily end up with the perfect predictor professor saying that in the scoring for this test, he will automatically fail anyone he predicts would throw the test in the previously described scenario. I don’t think there’s any point in time where making such a precommitment would have positive expected value. By the time I know it would have been useful, it’s already too late.
Do you also two box on Transparent Newcomb’s?
Edit: I think I was mistaken about what problem you were referring to. If I’m understanding the question correctly, yes I would, because until the scenario actually occurs I have no reason to suspect any precommitment I make is likely to bring about more favorable results. For any precommitment I could make, the scenario could always be inverted to punish that precommitment, so I’d just do what has the highest expected utility at the time at which I’m presented with the scenario. It would be different if my probability distribution on what precommitments would be useful weren’t totally flat.
As an aside, I’ll note that a lot of the solutions bandied around here to decision theory problems remind me of something from Magic: The Gathering which I took notice of back when I still followed it.
When I watched my friends play, one would frequently respond to another’s play with “Before you do that, I-” and use some card or ability to counter their opponent’s move. The rules of MTG let you do that sort of thing, but I always thought it was pretty silly, because they did not, in fact, have any idea that it would make sense to make that play until after seeing their opponent’s move. Once they see their opponent’s play, they get to retroactively decide what to do “before” their opponent can do it.
In real life, we don’t have that sort of privilege. If you’re in a Counterfactual Mugging scenario, for instance, you might be inclined to say “I ought to be the sort of person who would pay Omega, because if the coin had come up the other way, I would be making a lot of money now, so being that sort of person would have positive expected utility for this scenario.” But this is “Before you do that-” type reasoning. You could just as easily have ended up in a situation where Omega comes and tells you “I decided that if you were the sort of person who would not pay up in a Counterfactual Mugging scenario, I would give you a million dollars, but I’ve predicted that you would, so you get nothing.”
When you come up with a solution to an Omega-type problem involving some type of precommitment, it’s worth asking “would this precommitment have made sense when I was in a position of not knowing Omega existed, or having any idea what it would do even if it did exist?”
In real life, we sometimes have to make decisions dealing with agents who have some degree of predictive power with respect to our thought processes, but their motivations are generally not as arbitrary as those attributed to Omega in most hypotheticals.
Can you give a specific example of a bandied-around solution to a decision-theory problem where predictive power is necessary in order to implement that solution?
I suspect I disagree with you here—or, rather, I agree with the general principle you’ve articulated, but I suspect I disagree that it’s especially relevant to anything local—but it’s difficult to be sure without specifics.
With respect to the Counterfactual Mugging you reference in passing, for example, it seems enough to say “I ought to be the sort of person who would do whatever gets me positive expected utility”; I don’t have to specifically commit to pay or not pay. Isn’t it? But perhaps I’ve misunderstood the solution you’re rejecting.
Well, if your decision theory tells you you ought to be the sort of person who would pay up in a Counterfactual Mugging, because that gets you positive utility, then you could end up in with Omega coming and saying “I would have given you a million dollars if your decision theory said not to pay out in a counterfactual mugging, but since you would, you don’t get anything.”
When you know nothing about Omega, I don’t think there’s any positive expected utility in choosing to be the sort of person who would have positive expected utility in a Counterfactual Mugging scenario, because you have no reason to suspect it’s more likely than the inverted scenario where being that sort of person will get you negative utility. The probability distribution is flat, so the utilities cancel out.
Say Omega comes to you with a Counterfactual Mugging on Day 1. On Day 0, would you want to be the sort of person who pays out in a Counterfactual Mugging? No, because the probabilities of it being useful or harmful cancel out. On Day 1, when given the dilemma, do you want to be the sort of person who pays out in a Counterfactual Mugging? No, because now it only costs you money and you get nothing out of it.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Reasoning this way means, of course, that you don’t get the money in a situation where Omega would only pay you if it predicted you would pay up, but you do get the money in situations where Omega pays out only if you wouldn’t pay out. The latter possibility seems less salient from the “before you do that-” standpoint of a person contemplating a Counterfactual Mugging, but there’s no reason to assign it a lower probability before the fact. The best you can do is choose according to whatever has the highest expected utility at any given time.
Omega could also come and tell me “I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,” but I don’t let this influence my typing habits. You don’t want to alter your decision theories or general behavior in advance of specific events that are no more probable than their inversions.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Sure, I agree.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
Is this not true?
“I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,”
I agree that this is not something I can sensibly protect against. I’m not actually sure I would call it a decision theory problem at all.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
In the inversion I suggested to the Counterfactual Mugging, your payout is determined on the basis of whether you pay up in the Counterfactual Mugging. In the Counterfactual Mugging, Omega predicts whether you would pay out in the Counterfactual Mugging, and if you would, you get a 50% shot at a million dollars. In the inverted scenario, Omega predicts whether you would pay out in the Counterfactual Mugging scenario, and if you wouldn’t, you get a shot at a million dollars.
Being the sort of person who would pay out in a Counterfactual Mugging only brings positive expected utility if you expect the Counterfactual Mugging scenario to be more likely than the inverted Counterfactual Mugging scenario.
The inverted Counterfactual Mugging scenario, like the case where Omega rewards or punishes you based on your keyboard usage, isn’t exactly a decision theory problem, in that once it arises, you don’t get to make a decision, but it doesn’t need to be.
When the question is “should I be the sort of person who pays out in a Counterfactual Mugging?” if the chance of it being helpful is balanced out by an equal chance of it being harmful, then it doesn’t matter whether the situations that balance it out require you to make decisions at all, only that the expected utilities balance.
If you take as a premise “Omega simply doesn’t do that sort of thing, it only provides decision theory dilemmas where the results are dependent on how you would respond in this particular dilemma,” then our probability distribution is no longer flat, and being the sort of person who pays out in a Counterfactual Mugging scenario becomes utility maximizing. But this isn’t a premise we can take for granted. Omega is already posited as an entity which can judge your decision algorithms perfectly, and imposes dilemmas which are highly arbitrary.
Edit: I think I was mistaken about what problem you were referring to. If I’m understanding the question correctly, yes I would, because until the scenario actually occurs I have no reason to suspect any precommitment I make is likely to bring about more favorable results. For any precommitment I could make, the scenario could always be inverted to punish that precommitment, so I’d just do what has the highest expected utility at the time at which I’m presented with the scenario. It would be different if my probability distribution on what precommitments would be useful weren’t totally flat.
You don’t need a precommitment to make the correct choice. You just make it. That does happen to include one boxing on Transparent Newcomb’s (and conventional Newcomb’s, for the same reason). The ‘but what if someone punishes me for being the kind of person who makes this choice’ is a fully general excuse to not make rational choices. The reason why it is an invalid fully general excuse is because every scenario that can be contrived to result in ‘bad for you’ is one in which your rewards are determined by your behavior in an entirely different game to the one in question.
For example your “inverted Transparent Newcomb’s” gives you a bad outcome, but not because of your choice. It isn’t anything to do with a decision because you don’t get to make one. It is punishing you for your behavior in a completely different game.
Could you describe the Transparent Newcomb’s problem to me so I’m sure we’re on the same page?
“What if I face a scenario that punishes me for being the sort of person who makes this choice?” is not a fully general counterargument, it only applies in cases where the expected utilities of the scenarios cancel out.
If you’re the sort of person who won’t honor promises made under duress, and other people are sufficiently effective judges to recognize this, then you avoid people placing you under duress to extract promises from you. But supposing you’re captured by enemies in a war, and they say “We could let you go if you made some promises to help out our cause when you were free, but since we can’t trust you to keep them, we’re going to keep you locked up and torture you to make your country want to ransom you more.”
This doesn’t make the expected utilities of “Keep promises made under duress” vs. “Do not keep promises made under duress” cancel out, because you have an abundance of information with respect to how relatively likely these situations are.
Could you describe the Transparent Newcomb’s problem to me so I’m sure we’re on the same page?
Take a suitable description of Newcomb’s problem (you know, with Omega and boxes). Then make the boxes transparent. That is the extent of the difference. I assert that being able to see the money makes no difference to whether one should one box or two box (and also that one should one box).
Well, if you know advance that Omega is more likely to do this than it is to impose a dilemma where it will fill both boxes only if you two-box, then I’d agree that this is an appropriate solution.
I think that if in advance you have a flat probability distribution for what sort of Omega scenarios might occur (Omega is just as likely to fill both boxes only if you would two-box in the first scenario as it is to fill both boxes only if you would one-box,) then this solution doesn’t make sense.
In the transparent Newcomb’s problem, when both boxes are filled, does it benefit you to be the the sort of person who would one-box? No, because you get less money that way. If Omega is more likely to impose the transparent Newcomb’s problem than its inversion, then prior to Omega foisting the problem on you, it does benefit you to be the sort of person who would one-box (and you can’t change what sort of person you are mid-problem.)
If Omega only presents transparent Newcomb’s problems of the first sort, where the box containing more money is filled only if the person presented with the boxes would one-box, then situations where a person is presented with two transparent boxes of money and picks both will never arise. People who would one-box in the transparent Newcomb’s problem come out ahead.
If Omega is equally likely to present transparent Newcomb’s problems of the first sort, or inversions where Omega fills both boxes only for people it predicts would two-box in problems of the first sort, then two-boxers come out ahead, because they’re equally likely to get the contents of the box with more money, but always get the box with less money, while the one-boxers never do.
You can always contrive scenarios to reward or punish any particular decision theory. The Transparent Newcomb’s Problem rewards agents which one-box in the Transparent Newcomb’s Problem over agents which two-box, but unless this sort of problem is more likely to arise than ones which reward agents which two-box in Transparent Newcomb’s Problem over ones that one-box, that isn’t an an argument favoring decision theories which say you should one-box in Transparent Newcomb’s.
If you keep a flat probability distribution of what Omega would do to you prior to actually being put into a dilemma, expected-utility-maximizing still favors one-boxing in the opaque version of the dilemma (because based on the information available to you, you have to assign different probabilities to the opaque box containing money depending on whether you one-box or two-box,) but not one-boxing in the transparent version.
You can always contrive scenarios to reward or punish any particular decision theory. The Transparent Newcomb’s Problem rewards agents which one-box in the Transparent Newcomb’s Problem over agents which two-box, but unless this sort of problem is more likely to arise than ones which reward agents which two-box in Transparent Newcomb’s Problem over ones that one-box, that isn’t an an argument favoring decision theories which say you should one-box in Transparent Newcomb’s.
No, Transparent Newcomb’s, Newcomb’s and Prisoner’s Dilemma with full mutual knowledge don’t care what the decision algorithm is. They reward agents that take one box and mutually cooperate for no other reason than they decide to make the decision that benefits them.
You have presented a fully general argument for making bad choices. It can be used to reject “look both ways before crossing a road” just as well as it can be used to reject “get a million dollars by taking one box”. It should be applied to neither.
It’s not a fully general counterargument, it demands that you weigh the probabilities of potential outcomes.
If you look both ways at a crosswalk, you could be hit by a falling object that you would have avoided if you hadn’t paused in that location. Does that justify not looking both ways at a crosswalk? No, because the probability of something bad happening to you if you don’t look both ways at the crosswalk is higher than if you do.
You can always come up with absurd hypotheticals which would punish the behavior that would normally be rational in a particular situation. This doesn’t justify being paralyzed with indecision, the probabilities of the absurd hypotheticals materializing are miniscule. But the possibilities of absurd hypotheticals will tend to balance out other absurd hypotheticals.
Transparent Newcomb’s Problem is a problem that rewards agents which one-box in Transparent Newcomb’s Problem, via Omega predicting whether the agent one-boxes in Transparent Newcomb’s Problem and filling the boxes accordingly. Inverted Transparent Newcomb’s Problem is one that rewards agents that two-box in Transparent Newcomb’s Problem via Omega predicting whether the agent two-boxes in Transparent Newcomb’s Problem, and filling the boxes accordingly.
If one type of situation is more likely than the other, you adjust your expected utilities accordingly, just as you adjust your expected utility of looking both ways before you cross the street because you’re less likely to suffer an accident if you do than if you don’t.
Transparent Newcomb’s Problem is a problem that rewards agents which one-box in Transparent Newcomb’s Problem
Yes.
Inverted Transparent Newcomb’s Problem is one that rewards agents that two-box in Transparent Newcomb’s Problem via Omega predicting whether the agent two-boxes in Transparent Newcomb’s Problem, and filling the boxes accordingly.
That isn’t an ‘inversion’ but instead an entirely different problem in which agents are rewarded for things external to the problem.
There’s no reason an agent you interact with in a decision problem can’t respond to how it judges you would react to different decision problems.
Suppose Andy and Sandy are bitter rivals, and each wants the other to be socially isolated. Andy declares that he will only cooperate in Prisoner’s Dilemma type problems with people he predicts would cooperate with him, but not Sandy, while Sandy declares that she will only cooperate in Prisoner’s Dilemma type problems with people she predicts would cooperate with her, but not Andy. Both are highly reliable predictors of other people’s cooperation patterns.
If you end up in a Prisoner’s Dilemma type problem with Andy, it benefits you to be the sort of person who would cooperate with Andy, but not Sandy, and vice versa if you end up in a Prisoner’s Dilemma type problem with Sandy. If you might end up in a Prisoner’s Dilemma type problem with either of them, you have higher expected utility if you pick one in advance to cooperate with, because both would defect against an opportunist willing to cooperate with whichever one they ended up in a Prisoner’s Dilemma with first.
That isn’t an ‘inversion’ but instead an entirely different problem in which agents are rewarded for things external to the problem.
If you want to call it that, you may, but I don’t see that it makes a difference. If ending up in Transparent Newcomb’s Problem is no more likely than ending up in an entirely different problem which punishes agents for one-boxing in Transparent Newcomb’s Problem, then I don’t see that it’s advantageous to one-box in Transparent Newcomb’s Problem. You can draw a line between problems determined by factors external to the problem, and problems determined only by factors internal to the problem, but I don’t think this is a helpful distinction to apply here. What matters is which problems are more likely to occur and their utility payoffs.
In any case, I would honestly rather not continue this discussion with you, at least if TheOtherDave is still interested in continuing the discussion. I don’t have very high expectations of productivity from a discussion with someone who has such low expectations of my own reasoning as to repeatedly and erroneously declare that I’m calling up a fully general counterargument which could just as well be used to argue against looking both ways at a crosswalk. If possible, I would much rather discuss this with someone who’s prepared to operate under the presumption that I’m willing and able to be reasonable.
(I haven’t followed the discussion, so might be missing the point.)
If ending up in Transparent Newcomb’s Problem is no more likely than ending up in an entirely different problem which punishes agents for one-boxing in Transparent Newcomb’s Problem, then I don’t see that it’s advantageous to one-box in Transparent Newcomb’s Problem.
If you are actually in problem A, it’s advantageous to be solving problem A, even if there is another problem B in which you could have much more likely ended up. You are in problem A by stipulation. At the point where you’ve landed in the hypothetical of solving problem A, discussing problem B is a wrong thing to do, it interferes with trying to understand problem A. The difficulty of telling problem A from problem B is a separate issue that’s usually ruled out by hypothesis. We might discuss this issue, but that would be a problem C that shouldn’t be confused with problems A and B, where by hypothesis you know that you are dealing with problems A and B. Don’t fight the hypothetical.
In the case of Transparent Newcomb’s though, if you’re actually in the problem, then you can already see either that both boxes contain money, or that one of them doesn’t. If Omega only fills the second box, which contains more money, if you would one-box, then by the time you find yourself in the problem, whether you would one-box or two-box in Transparent Newcomb’s has already had its payoff.
If I would two-box in a situation where I see two transparent boxes which both contain money, that ensures that I won’t find myself in a situation where Omega lets me pick whether to one-box or two-box, but only fills both boxes if I would one-box. On the other hand, A person who one-boxes in that situation could not find themself in a situation where they can pick one or both of two filled boxes, where Omega would only fill both boxes if they would two-box in the original scenario.
So it seems to me that if I follow the principle of solving whatever situation I’m in according to maximum expected utility, then unless the Transparent Newcomb’s Problem is more probable, I will become the sort of person who can’t end up in Transparent Newcomb’s problems with a chance to one-box for large amounts of money, but can end up in the inverted situation which rewards two-boxing, for more money. I don’t have the choice of being the sort of person who gets rewarded by both scenarios, just as I don’t have the choice of being someone who both Andy and Sandy will cooperate with.
I agree that a one-boxer comes out ahead in Transparent Newcomb’s, but I don’t think it follows that I should one-box in Transparent Newcomb’s, because I don’t think having a decision theory which results in better payouts in this particular decision theory problem results in higher utility in general. I think that I “should” be a person who one-boxes in Transparent Newcomb’s in the same sense that I “should” be someone who doesn’t type between 10:00-11:00 on a Sunday if I happen to be in a world where Omega has, unbeknownst to anyone, arranged to rob me if I do. In both cases I’ve lucked into payouts due to a decision process which I couldn’t reasonably have expected to improve my utility.
I agree that a one-boxer comes out ahead in Transparent Newcomb’s, but I don’t think it follows that I should one-box in Transparent Newcomb’s, because I don’t think having a decision theory which results in better payouts in this particular decision theory problem results in higher utility in general.
We are not discussing what to do “in general”, or the algorithms of a general “I” that should or shouldn’t have the property of behaving a certain way in certain problems, we are discussing what should be done in this particular problem, where we might as well assume that there is no other possible problem, and all utility in the world only comes from this one instance of this problem. The focus is on this problem only, and no role is played by the uncertainty about which problem we are solving, or by the possibility that there might be other problems. If you additionally want to avoid logical impossibility introduced by some of the possible decisions, permit a very low probability that either of the relevant outcomes can occur anyway.
If you allow yourself to consider alternative situations, or other applications of the same decision algorithm, you are solving a different problem, a problem that involves tradeoffs between these situations. You need to be clear on which problem you are considering, whether it’s a single isolated problem, as is usual for thought experiments, or a bigger problem. If it’s a bigger problem, that needs to be prominently stipulated somewhere, or people will assume that it’s otherwise and you’ll talk past each other.
It seems as if you currently believe that the correct solution for isolated Transparent Newcomb’s is one-boxing, but the correct solution in the context of the possibility of other problems is two-boxing. Is it so? (You seem to understand “I’m in Transparent Newcomb’s problem” incorrectly, which further motivates fighting the hypothetical, suggesting that for the general player that has other problems on its plate two-boxing is better, which is not so, but it’s a separate issue, so let’s settle the problem statement first.)
It seems as if you currently believe that the correct solution for isolated Transparent Newcomb’s is one-boxing, but the correct solution in the context of the possibility of other problems is two-boxing. Is it so?
Yes.
I don’t think that the most advantageous solution for isolated Transparent Newcomb’s is likely to be a very useful question though.
I don’t think it’s possible to have a general case decision theory which gets the best possible results for every situation (see the Andy and Sandy example, where getting good results for one prisoner’s dilemma necessitates getting bad results from the other, so any decision theory wins in at most one of the two.)
That being the case, I don’t think that a goal of winning in Transparent Newcomb’s Problem is a very meaningful one for a decision theory. The way I see it, it seems like focusing on coming out ahead in Sandy prisoner’s dilemmas, while disregarding the relative likelihoods of ending up in a dilemma with Andy or Sandy, and assuming that if you ended up in an Andy prisoner dilemma you could use the same decision process to come out ahead in that too.
If possible, I would much rather discuss this with someone who’s prepared to operate under the presumption that I’m willing and able to be reasonable.
Don’t confuse an intuition aid that failed to help you with a personal insult. Apart from making you feel bad it’ll ensure you miss the point. Hopefully Vladimir’s explanation will be more successful.
I didn’t take it as a personal insult, I took it as a mistaken interpretation of my own argument which would have been very unlikely to come from someone who expected me to have reasoned through my position competently and was making a serious effort to understand it. So while it was not a personal insult, it was certainly insulting.
I may be failing to understand your position, and rejecting it only due to a misunderstanding, but from where I stand, your assertion makes it appear tremendously unlikely that you understand mine.
If you think that my argument generalizes to justifying any bad decision, including cases like not looking both ways when I cross the street, when I say otherwise, it would help if you would explain why you think it generalizes in this way in spite of the reasons I’ve given for believing otherwise, rather than simply repeating the assertion without acknowledging them, otherwise it looks like you’re either not making much effort to comprehend my position, or don’t care much about explaining yours, and are only interested in contradicting someone you think is wrong.
Edit: I would prefer you not respond to this comment, and in any case I don’t intend to respond to a response, because I don’t expect this conversation to be productive, and I hate going to bed wondering how I’m going to continue tomorrow what I expect to be a fruitless conversation.
No, I don’t, since you have a time-turner. (To be clear, non-hypothetical-me wouldn’t hit non-hypothetical-you either.) I would also one-box if I thought that Omega’s predictive power was evidence that it might have a time turner or some other way of affecting the past. I still don’t think that’s relevant when there’s no reverse causality.
Back to Newcomb’s problem: Say that brown-haired people almost always one-box, and people with other hair colors almost always two-box. Omega predicts on the basis of hair color: both boxes are filled iff you have brown hair. I’d two-box, even though I have brown hair. It would be logically inconsistent for me to find that one of the boxes is empty, since everyone with brown hair has both boxes filled. But this could be true of any attribute Omega uses to predict.
I agree that changing my decision conveys information about what is in the boxes and changes my guess of what is in the boxes… but doesn’t change the boxes.
Back to Newcomb’s problem: Say that brown-haired people almost always one-box, and people with other hair colors almost always two-box. Omega predicts on the basis of hair color: both boxes are filled iff you have brown hair. I’d two-box, even though I have brown hair. It would be logically inconsistent for me to find that one of the boxes is empty, since everyone with brown hair has both boxes filled. But this could be true of any attribute Omega uses to predict.
If the agent filling the boxes follows a consistent, predictable pattern you’re outside of, you can certainly use that information to do this. In Newcomb’s Problem though, Omega follows a consistent, predictable pattern you’re inside of. It’s logically inconsistent for you to two box and find they both contain money, or pick one box and find it’s empty.
I agree that changing my decision conveys information about what is in the boxes and changes my guess of what is in the boxes… but doesn’t change the boxes.
Why is whether your decision actually changes the boxes important to you? If you know that picking one box will result in your receiving a million dollars, and picking two boxes will result in getting a thousand dollars, do you have any concern that overrides making the choice that you expect to make you more money?
A decision process of “at all times, do whatever I expect to have the best results” will, at worst, reduce to exactly the same behavior as “at all times, do whatever I think will have a causal relationship with the best results.” In some cases, such as Newcomb’s problem, it has better results. What do you think the concern with causality actually does for you?
We don’t always agree here on what decision theories get the best results (as you can see by observing the offshoot of this conversation between Wedrifid and myself,) but what we do generally agree on here is that the quality of decision theories is determined by their results. If you argue yourself into a decision theory that doesn’t serve you well, you’ve only managed to shoot yourself in the foot.
Why is whether your decision actually changes the boxes important to you?
[....]
If you argue yourself into a decision theory that doesn’t serve you well, you’ve only managed to shoot yourself in the foot.
In the absence of my decision affecting the boxes, taking one box and leaving $1000 on the table still looks like shooting myself in the foot. (Of course if I had the ability to precommit to one-box I would—so, okay, if Omega ever asks me this I will take one box. But if Omega asked me to make a decision after filling the boxes and before I’d made a precommitment… still two boxes.)
I think I’m going to back out of this discussion until I understand decision theory a bit better.
I think I’m going to back out of this discussion until I understand decision theory a bit better.
Feel free. You can revisit this conversation any time you feel like it. Discussion threads never really die here, there’s no community norm against replying to comments long after they’re posted.
To be properly isomorphic to the Newcomb’s problem, the chance of the predictor being wrong should approximate to zero.
If I thought that the chance of my friend’s mother being wrong approximated to zero, I would of course choose to one-box. If I expected her to be an imperfect predictor who assumed I would behave as if I were in the real Newcomb’s problem with a perfect predictor, then I would choose to two-box.
In Newcomb’s Problem, if you choose on the basis of which choice is consistent with a higher expected return, then you would choose to one-box. You know that your choice doesn’t cause the box to be filled, but given the knowledge that whether the money is in the box or not is contingent on a perfect predictor’s assessment of whether or not you were likely to one-box, you should assign different probabilities to the box containing the money depending on whether you one-box or two-box. Since your own mental disposition is evidence of whether the money is in the box or not, you can behave as if the contents were determined by your choice.
Hm, I think I still don’t understand the one-box perspective, then. Are you saying that if the predictor is wrong with probability p, you would take two-boxes for high p and one box for a sufficiently small p (or just for p=0)? What changes as p shrinks?
Or what if Omega/Ann’s mom is a perfect predictor, but for a random 1% of the time decides to fill the boxes as if it made the opposite prediction, just to mess with you? If you one-box for p=0, you should believe that taking one box is correct (and generates $1 million more) in 99% of cases and that two boxes is correct (and generates $1000 more) in 1% of cases. So taking one box should still have a far higher expected value. But the perfect predictor who sometimes pretends to be wrong behaves exactly the same as an imperfect predictor who is wrong 1% of the time.
You choose the boxes according to the expected value of each box choice. For a 99% accurate predictor, the expected value of one-boxing is $990,000,000 (you get a billion 99% of the time, and nothing 1% of the time,) while the expected value of two-boxing is $10,001,000 (you get a thousand 99% of the time, and one billion and one thousand 1% of the time.)
The difference between this scenario and the one you posited before, where Ann’s mom makes her prediction by reading your philosophy essays, is that she’s presumably predicting on the basis of how she would expect you to choose if you were playing Omega. If you’re playing against an agent who you know will fill the boxes according to how you would choose if you were playing Omega (we’ll call it Omega-1,) then you should always two-box (if you would one-box against Omega, both boxes will contain money, so you get the contents of both. If you would two-box against Omega, only one box would contain money, and if you one-box you’ll get the empty one.)
An imperfect predictor with random error is a different proposition from an imperfect predictor with nonrandom error.
Of course, if I were dealing with this dilemma in real life, my choice would be heavily influenced by considerations such as how likely it is that Ann’s mom really has billions of dollars to give away.
Ok, but what if Ann’s mom is right 99% of the time about how you would choose when playing her?
I agree that one-boxers make more money, with the numbers you used, but I don’t think that those are the appropriate expected values to consider. Conditional on the fact that the boxes have already been filled, two-boxing has a $1000 higher expected value. If I know only one box is filled, I should take both. If I know both boxes are filled, I should take both. If I know I’m in one of those situations but not sure of which it is, I should still take both.
Another analogous situation would be that you walk into an exam, and the professor (who is a perfect or near-perfect predictor) announces that he has written down a list of people whom he has predicted will get fewer than half the questions right. If you are on that list, he will add 100 points to your score at the end. The people who get fewer than half of the questions right get higher scores, but you should still try to get questions right on the test… right? If not, does the answer change if the professor posts the list on the board?
I still think I’m missing something, since a lot of people have thought carefully about this and come to a different conclusion from me, but I’m still not sure what it is. :/
You are focusing too much on the “already have been filled”, as if the particular time of your particular decision is relevant. But if your decision isn’t random (and yours isn’t), then any individual decision is dependent on the decision algorithm you follow—and can be calculated in exactly the same manner, regardless of time. Therefore in a sense your decision has been made BEFORE the filling of the boxes, and can affect their contents.
You may consider it easier to wrap your head around this if you think of the boxes being filled according to what result the decision theory you currently have would return in the situation, instead of what decision you’ll make in the future. That helps keep in mind that causality still travels only one direction, but that a good predictor simply knows the decision you’ll make before you make it and can act accordingly.
I would one-box. I gave the relevant numbers on this in my previous comment; one-boxing has an expected value of $990,000,000 to the expected $10,001,000 if you two-box.
When you’re dealing with a problem involving an effective predictor of your own mental processes (it’s not necessary for such a predictor to be perfect for this reasoning to become salient, it just makes the problems simpler,) your expectation of what the predictor will do or already have done will be at least partly dependent on what you intend to do yourself. You know that either the opaque box is filled, or it is not, but the probability you assign to the box being filled depends on whether you intend to open it or not.
Let’s try a somewhat different scenario. Suppose I have a time machine that allows me to travel back a day in the past. Doing so creates a stable time loop, like the time turners in Harry Potter or HPMoR (on a side note, our current models of relativity suggest that such loops are possible, if very difficult to contrive.) You’re angry at me because I’ve insulted your hypothetical scenario, and are considering hitting me in retaliation. But you happen to know that I retaliate against people who hit me by going back in time and stealing from them, which I always get away with due to having perfect alibis (the police don’t believe in my time machine.) You do not know whether I’ve stolen from you or not, but if I have, it’s already happened. You would feel satisfied by hitting me, but it’s not worth being stolen from. Do you choose to hit me or not?
If the professor is a perfect predictor, then I would deliberately get most of the problems wrong, thereby all but guaranteeing a score of over 100 points. I would have to be very confident that I would get a score below fifty even if I weren’t trying to on purpose before trying to get all the questions right would give me a higher expected score than trying to get most of the questions wrong.
If the professor posts the list on the board, then of course it should affect the answer. If my name isn’t on the list, then he’s not going to add the 100 points to my test in any case, so my only recourse to maximizing my grade is to try my best on the test. If my name is on the list, then he’s already predicted that I’m going to score below 50, so whether he’s a perfect predictor or not, I should try to do well so that he’s adding 100 points to as high a score as I can manage.
The difference between the scenario where he writes the names on the board and the scenario where he doesn’t is that in the former, my expectations of his actions don’t vary according to my own, whereas in the latter, they do.
I believe you are making a mistake. Specifically, you are implementing a decision algorithm that ensures that “you lose” is a correct self fulfilling prophecy (in fact you ensure that it is the only valid prediction he could make). I would throw the test (score in the 40s) even when my name is not on the list.
Do you also two box on Transparent Newcomb’s?
If I were in a position to predict that this were the sort of thing the professor might do, then I would precommit to throwing the test should he implement such a procedure. But you could just as easily end up with the perfect predictor professor saying that in the scoring for this test, he will automatically fail anyone he predicts would throw the test in the previously described scenario. I don’t think there’s any point in time where making such a precommitment would have positive expected value. By the time I know it would have been useful, it’s already too late.
Edit: I think I was mistaken about what problem you were referring to. If I’m understanding the question correctly, yes I would, because until the scenario actually occurs I have no reason to suspect any precommitment I make is likely to bring about more favorable results. For any precommitment I could make, the scenario could always be inverted to punish that precommitment, so I’d just do what has the highest expected utility at the time at which I’m presented with the scenario. It would be different if my probability distribution on what precommitments would be useful weren’t totally flat.
As an aside, I’ll note that a lot of the solutions bandied around here to decision theory problems remind me of something from Magic: The Gathering which I took notice of back when I still followed it.
When I watched my friends play, one would frequently respond to another’s play with “Before you do that, I-” and use some card or ability to counter their opponent’s move. The rules of MTG let you do that sort of thing, but I always thought it was pretty silly, because they did not, in fact, have any idea that it would make sense to make that play until after seeing their opponent’s move. Once they see their opponent’s play, they get to retroactively decide what to do “before” their opponent can do it.
In real life, we don’t have that sort of privilege. If you’re in a Counterfactual Mugging scenario, for instance, you might be inclined to say “I ought to be the sort of person who would pay Omega, because if the coin had come up the other way, I would be making a lot of money now, so being that sort of person would have positive expected utility for this scenario.” But this is “Before you do that-” type reasoning. You could just as easily have ended up in a situation where Omega comes and tells you “I decided that if you were the sort of person who would not pay up in a Counterfactual Mugging scenario, I would give you a million dollars, but I’ve predicted that you would, so you get nothing.”
When you come up with a solution to an Omega-type problem involving some type of precommitment, it’s worth asking “would this precommitment have made sense when I was in a position of not knowing Omega existed, or having any idea what it would do even if it did exist?”
In real life, we sometimes have to make decisions dealing with agents who have some degree of predictive power with respect to our thought processes, but their motivations are generally not as arbitrary as those attributed to Omega in most hypotheticals.
Can you give a specific example of a bandied-around solution to a decision-theory problem where predictive power is necessary in order to implement that solution?
I suspect I disagree with you here—or, rather, I agree with the general principle you’ve articulated, but I suspect I disagree that it’s especially relevant to anything local—but it’s difficult to be sure without specifics.
With respect to the Counterfactual Mugging you reference in passing, for example, it seems enough to say “I ought to be the sort of person who would do whatever gets me positive expected utility”; I don’t have to specifically commit to pay or not pay. Isn’t it? But perhaps I’ve misunderstood the solution you’re rejecting.
Well, if your decision theory tells you you ought to be the sort of person who would pay up in a Counterfactual Mugging, because that gets you positive utility, then you could end up in with Omega coming and saying “I would have given you a million dollars if your decision theory said not to pay out in a counterfactual mugging, but since you would, you don’t get anything.”
When you know nothing about Omega, I don’t think there’s any positive expected utility in choosing to be the sort of person who would have positive expected utility in a Counterfactual Mugging scenario, because you have no reason to suspect it’s more likely than the inverted scenario where being that sort of person will get you negative utility. The probability distribution is flat, so the utilities cancel out.
Say Omega comes to you with a Counterfactual Mugging on Day 1. On Day 0, would you want to be the sort of person who pays out in a Counterfactual Mugging? No, because the probabilities of it being useful or harmful cancel out. On Day 1, when given the dilemma, do you want to be the sort of person who pays out in a Counterfactual Mugging? No, because now it only costs you money and you get nothing out of it.
So there’s no point in time where deciding “I should be the sort of person who pays out in a Counterfactual Mugging” has positive expected utility.
Reasoning this way means, of course, that you don’t get the money in a situation where Omega would only pay you if it predicted you would pay up, but you do get the money in situations where Omega pays out only if you wouldn’t pay out. The latter possibility seems less salient from the “before you do that-” standpoint of a person contemplating a Counterfactual Mugging, but there’s no reason to assign it a lower probability before the fact. The best you can do is choose according to whatever has the highest expected utility at any given time.
Omega could also come and tell me “I decided that I would steal all your money if you hit the S key on your keyboard between 10:00-11:00 am on a Sunday, and you just did,” but I don’t let this influence my typing habits. You don’t want to alter your decision theories or general behavior in advance of specific events that are no more probable than their inversions.
Sure, I agree.
What I’m suggesting is that “I should be the sort of person who does the thing that has positive expected utility” causes me to pay out in a Counterfactual Mugging, and causes me to not pay out in a Counterfactual Antimugging, without requiring any prophecy. And that as far as I know, this is representative of the locally bandied-around solutions to decision-theory problems.
Is this not true?
I agree that this is not something I can sensibly protect against. I’m not actually sure I would call it a decision theory problem at all.
In the inversion I suggested to the Counterfactual Mugging, your payout is determined on the basis of whether you pay up in the Counterfactual Mugging. In the Counterfactual Mugging, Omega predicts whether you would pay out in the Counterfactual Mugging, and if you would, you get a 50% shot at a million dollars. In the inverted scenario, Omega predicts whether you would pay out in the Counterfactual Mugging scenario, and if you wouldn’t, you get a shot at a million dollars.
Being the sort of person who would pay out in a Counterfactual Mugging only brings positive expected utility if you expect the Counterfactual Mugging scenario to be more likely than the inverted Counterfactual Mugging scenario.
The inverted Counterfactual Mugging scenario, like the case where Omega rewards or punishes you based on your keyboard usage, isn’t exactly a decision theory problem, in that once it arises, you don’t get to make a decision, but it doesn’t need to be.
When the question is “should I be the sort of person who pays out in a Counterfactual Mugging?” if the chance of it being helpful is balanced out by an equal chance of it being harmful, then it doesn’t matter whether the situations that balance it out require you to make decisions at all, only that the expected utilities balance.
If you take as a premise “Omega simply doesn’t do that sort of thing, it only provides decision theory dilemmas where the results are dependent on how you would respond in this particular dilemma,” then our probability distribution is no longer flat, and being the sort of person who pays out in a Counterfactual Mugging scenario becomes utility maximizing. But this isn’t a premise we can take for granted. Omega is already posited as an entity which can judge your decision algorithms perfectly, and imposes dilemmas which are highly arbitrary.
You don’t need a precommitment to make the correct choice. You just make it. That does happen to include one boxing on Transparent Newcomb’s (and conventional Newcomb’s, for the same reason). The ‘but what if someone punishes me for being the kind of person who makes this choice’ is a fully general excuse to not make rational choices. The reason why it is an invalid fully general excuse is because every scenario that can be contrived to result in ‘bad for you’ is one in which your rewards are determined by your behavior in an entirely different game to the one in question.
For example your “inverted Transparent Newcomb’s” gives you a bad outcome, but not because of your choice. It isn’t anything to do with a decision because you don’t get to make one. It is punishing you for your behavior in a completely different game.
Could you describe the Transparent Newcomb’s problem to me so I’m sure we’re on the same page?
“What if I face a scenario that punishes me for being the sort of person who makes this choice?” is not a fully general counterargument, it only applies in cases where the expected utilities of the scenarios cancel out.
If you’re the sort of person who won’t honor promises made under duress, and other people are sufficiently effective judges to recognize this, then you avoid people placing you under duress to extract promises from you. But supposing you’re captured by enemies in a war, and they say “We could let you go if you made some promises to help out our cause when you were free, but since we can’t trust you to keep them, we’re going to keep you locked up and torture you to make your country want to ransom you more.”
This doesn’t make the expected utilities of “Keep promises made under duress” vs. “Do not keep promises made under duress” cancel out, because you have an abundance of information with respect to how relatively likely these situations are.
Take a suitable description of Newcomb’s problem (you know, with Omega and boxes). Then make the boxes transparent. That is the extent of the difference. I assert that being able to see the money makes no difference to whether one should one box or two box (and also that one should one box).
Well, if you know advance that Omega is more likely to do this than it is to impose a dilemma where it will fill both boxes only if you two-box, then I’d agree that this is an appropriate solution.
I think that if in advance you have a flat probability distribution for what sort of Omega scenarios might occur (Omega is just as likely to fill both boxes only if you would two-box in the first scenario as it is to fill both boxes only if you would one-box,) then this solution doesn’t make sense.
In the transparent Newcomb’s problem, when both boxes are filled, does it benefit you to be the the sort of person who would one-box? No, because you get less money that way. If Omega is more likely to impose the transparent Newcomb’s problem than its inversion, then prior to Omega foisting the problem on you, it does benefit you to be the sort of person who would one-box (and you can’t change what sort of person you are mid-problem.)
If Omega only presents transparent Newcomb’s problems of the first sort, where the box containing more money is filled only if the person presented with the boxes would one-box, then situations where a person is presented with two transparent boxes of money and picks both will never arise. People who would one-box in the transparent Newcomb’s problem come out ahead.
If Omega is equally likely to present transparent Newcomb’s problems of the first sort, or inversions where Omega fills both boxes only for people it predicts would two-box in problems of the first sort, then two-boxers come out ahead, because they’re equally likely to get the contents of the box with more money, but always get the box with less money, while the one-boxers never do.
You can always contrive scenarios to reward or punish any particular decision theory. The Transparent Newcomb’s Problem rewards agents which one-box in the Transparent Newcomb’s Problem over agents which two-box, but unless this sort of problem is more likely to arise than ones which reward agents which two-box in Transparent Newcomb’s Problem over ones that one-box, that isn’t an an argument favoring decision theories which say you should one-box in Transparent Newcomb’s.
If you keep a flat probability distribution of what Omega would do to you prior to actually being put into a dilemma, expected-utility-maximizing still favors one-boxing in the opaque version of the dilemma (because based on the information available to you, you have to assign different probabilities to the opaque box containing money depending on whether you one-box or two-box,) but not one-boxing in the transparent version.
No, Transparent Newcomb’s, Newcomb’s and Prisoner’s Dilemma with full mutual knowledge don’t care what the decision algorithm is. They reward agents that take one box and mutually cooperate for no other reason than they decide to make the decision that benefits them.
You have presented a fully general argument for making bad choices. It can be used to reject “look both ways before crossing a road” just as well as it can be used to reject “get a million dollars by taking one box”. It should be applied to neither.
It’s not a fully general counterargument, it demands that you weigh the probabilities of potential outcomes.
If you look both ways at a crosswalk, you could be hit by a falling object that you would have avoided if you hadn’t paused in that location. Does that justify not looking both ways at a crosswalk? No, because the probability of something bad happening to you if you don’t look both ways at the crosswalk is higher than if you do.
You can always come up with absurd hypotheticals which would punish the behavior that would normally be rational in a particular situation. This doesn’t justify being paralyzed with indecision, the probabilities of the absurd hypotheticals materializing are miniscule. But the possibilities of absurd hypotheticals will tend to balance out other absurd hypotheticals.
Transparent Newcomb’s Problem is a problem that rewards agents which one-box in Transparent Newcomb’s Problem, via Omega predicting whether the agent one-boxes in Transparent Newcomb’s Problem and filling the boxes accordingly. Inverted Transparent Newcomb’s Problem is one that rewards agents that two-box in Transparent Newcomb’s Problem via Omega predicting whether the agent two-boxes in Transparent Newcomb’s Problem, and filling the boxes accordingly.
If one type of situation is more likely than the other, you adjust your expected utilities accordingly, just as you adjust your expected utility of looking both ways before you cross the street because you’re less likely to suffer an accident if you do than if you don’t.
Yes.
That isn’t an ‘inversion’ but instead an entirely different problem in which agents are rewarded for things external to the problem.
There’s no reason an agent you interact with in a decision problem can’t respond to how it judges you would react to different decision problems.
Suppose Andy and Sandy are bitter rivals, and each wants the other to be socially isolated. Andy declares that he will only cooperate in Prisoner’s Dilemma type problems with people he predicts would cooperate with him, but not Sandy, while Sandy declares that she will only cooperate in Prisoner’s Dilemma type problems with people she predicts would cooperate with her, but not Andy. Both are highly reliable predictors of other people’s cooperation patterns.
If you end up in a Prisoner’s Dilemma type problem with Andy, it benefits you to be the sort of person who would cooperate with Andy, but not Sandy, and vice versa if you end up in a Prisoner’s Dilemma type problem with Sandy. If you might end up in a Prisoner’s Dilemma type problem with either of them, you have higher expected utility if you pick one in advance to cooperate with, because both would defect against an opportunist willing to cooperate with whichever one they ended up in a Prisoner’s Dilemma with first.
If you want to call it that, you may, but I don’t see that it makes a difference. If ending up in Transparent Newcomb’s Problem is no more likely than ending up in an entirely different problem which punishes agents for one-boxing in Transparent Newcomb’s Problem, then I don’t see that it’s advantageous to one-box in Transparent Newcomb’s Problem. You can draw a line between problems determined by factors external to the problem, and problems determined only by factors internal to the problem, but I don’t think this is a helpful distinction to apply here. What matters is which problems are more likely to occur and their utility payoffs.
In any case, I would honestly rather not continue this discussion with you, at least if TheOtherDave is still interested in continuing the discussion. I don’t have very high expectations of productivity from a discussion with someone who has such low expectations of my own reasoning as to repeatedly and erroneously declare that I’m calling up a fully general counterargument which could just as well be used to argue against looking both ways at a crosswalk. If possible, I would much rather discuss this with someone who’s prepared to operate under the presumption that I’m willing and able to be reasonable.
(I haven’t followed the discussion, so might be missing the point.)
If you are actually in problem A, it’s advantageous to be solving problem A, even if there is another problem B in which you could have much more likely ended up. You are in problem A by stipulation. At the point where you’ve landed in the hypothetical of solving problem A, discussing problem B is a wrong thing to do, it interferes with trying to understand problem A. The difficulty of telling problem A from problem B is a separate issue that’s usually ruled out by hypothesis. We might discuss this issue, but that would be a problem C that shouldn’t be confused with problems A and B, where by hypothesis you know that you are dealing with problems A and B. Don’t fight the hypothetical.
In the case of Transparent Newcomb’s though, if you’re actually in the problem, then you can already see either that both boxes contain money, or that one of them doesn’t. If Omega only fills the second box, which contains more money, if you would one-box, then by the time you find yourself in the problem, whether you would one-box or two-box in Transparent Newcomb’s has already had its payoff.
If I would two-box in a situation where I see two transparent boxes which both contain money, that ensures that I won’t find myself in a situation where Omega lets me pick whether to one-box or two-box, but only fills both boxes if I would one-box. On the other hand, A person who one-boxes in that situation could not find themself in a situation where they can pick one or both of two filled boxes, where Omega would only fill both boxes if they would two-box in the original scenario.
So it seems to me that if I follow the principle of solving whatever situation I’m in according to maximum expected utility, then unless the Transparent Newcomb’s Problem is more probable, I will become the sort of person who can’t end up in Transparent Newcomb’s problems with a chance to one-box for large amounts of money, but can end up in the inverted situation which rewards two-boxing, for more money. I don’t have the choice of being the sort of person who gets rewarded by both scenarios, just as I don’t have the choice of being someone who both Andy and Sandy will cooperate with.
I agree that a one-boxer comes out ahead in Transparent Newcomb’s, but I don’t think it follows that I should one-box in Transparent Newcomb’s, because I don’t think having a decision theory which results in better payouts in this particular decision theory problem results in higher utility in general. I think that I “should” be a person who one-boxes in Transparent Newcomb’s in the same sense that I “should” be someone who doesn’t type between 10:00-11:00 on a Sunday if I happen to be in a world where Omega has, unbeknownst to anyone, arranged to rob me if I do. In both cases I’ve lucked into payouts due to a decision process which I couldn’t reasonably have expected to improve my utility.
We are not discussing what to do “in general”, or the algorithms of a general “I” that should or shouldn’t have the property of behaving a certain way in certain problems, we are discussing what should be done in this particular problem, where we might as well assume that there is no other possible problem, and all utility in the world only comes from this one instance of this problem. The focus is on this problem only, and no role is played by the uncertainty about which problem we are solving, or by the possibility that there might be other problems. If you additionally want to avoid logical impossibility introduced by some of the possible decisions, permit a very low probability that either of the relevant outcomes can occur anyway.
If you allow yourself to consider alternative situations, or other applications of the same decision algorithm, you are solving a different problem, a problem that involves tradeoffs between these situations. You need to be clear on which problem you are considering, whether it’s a single isolated problem, as is usual for thought experiments, or a bigger problem. If it’s a bigger problem, that needs to be prominently stipulated somewhere, or people will assume that it’s otherwise and you’ll talk past each other.
It seems as if you currently believe that the correct solution for isolated Transparent Newcomb’s is one-boxing, but the correct solution in the context of the possibility of other problems is two-boxing. Is it so? (You seem to understand “I’m in Transparent Newcomb’s problem” incorrectly, which further motivates fighting the hypothetical, suggesting that for the general player that has other problems on its plate two-boxing is better, which is not so, but it’s a separate issue, so let’s settle the problem statement first.)
Yes.
I don’t think that the most advantageous solution for isolated Transparent Newcomb’s is likely to be a very useful question though.
I don’t think it’s possible to have a general case decision theory which gets the best possible results for every situation (see the Andy and Sandy example, where getting good results for one prisoner’s dilemma necessitates getting bad results from the other, so any decision theory wins in at most one of the two.)
That being the case, I don’t think that a goal of winning in Transparent Newcomb’s Problem is a very meaningful one for a decision theory. The way I see it, it seems like focusing on coming out ahead in Sandy prisoner’s dilemmas, while disregarding the relative likelihoods of ending up in a dilemma with Andy or Sandy, and assuming that if you ended up in an Andy prisoner dilemma you could use the same decision process to come out ahead in that too.
Don’t confuse an intuition aid that failed to help you with a personal insult. Apart from making you feel bad it’ll ensure you miss the point. Hopefully Vladimir’s explanation will be more successful.
I didn’t take it as a personal insult, I took it as a mistaken interpretation of my own argument which would have been very unlikely to come from someone who expected me to have reasoned through my position competently and was making a serious effort to understand it. So while it was not a personal insult, it was certainly insulting.
I may be failing to understand your position, and rejecting it only due to a misunderstanding, but from where I stand, your assertion makes it appear tremendously unlikely that you understand mine.
If you think that my argument generalizes to justifying any bad decision, including cases like not looking both ways when I cross the street, when I say otherwise, it would help if you would explain why you think it generalizes in this way in spite of the reasons I’ve given for believing otherwise, rather than simply repeating the assertion without acknowledging them, otherwise it looks like you’re either not making much effort to comprehend my position, or don’t care much about explaining yours, and are only interested in contradicting someone you think is wrong.
Edit: I would prefer you not respond to this comment, and in any case I don’t intend to respond to a response, because I don’t expect this conversation to be productive, and I hate going to bed wondering how I’m going to continue tomorrow what I expect to be a fruitless conversation.
No, I don’t, since you have a time-turner. (To be clear, non-hypothetical-me wouldn’t hit non-hypothetical-you either.) I would also one-box if I thought that Omega’s predictive power was evidence that it might have a time turner or some other way of affecting the past. I still don’t think that’s relevant when there’s no reverse causality.
Back to Newcomb’s problem: Say that brown-haired people almost always one-box, and people with other hair colors almost always two-box. Omega predicts on the basis of hair color: both boxes are filled iff you have brown hair. I’d two-box, even though I have brown hair. It would be logically inconsistent for me to find that one of the boxes is empty, since everyone with brown hair has both boxes filled. But this could be true of any attribute Omega uses to predict.
I agree that changing my decision conveys information about what is in the boxes and changes my guess of what is in the boxes… but doesn’t change the boxes.
If the agent filling the boxes follows a consistent, predictable pattern you’re outside of, you can certainly use that information to do this. In Newcomb’s Problem though, Omega follows a consistent, predictable pattern you’re inside of. It’s logically inconsistent for you to two box and find they both contain money, or pick one box and find it’s empty.
Why is whether your decision actually changes the boxes important to you? If you know that picking one box will result in your receiving a million dollars, and picking two boxes will result in getting a thousand dollars, do you have any concern that overrides making the choice that you expect to make you more money?
A decision process of “at all times, do whatever I expect to have the best results” will, at worst, reduce to exactly the same behavior as “at all times, do whatever I think will have a causal relationship with the best results.” In some cases, such as Newcomb’s problem, it has better results. What do you think the concern with causality actually does for you?
We don’t always agree here on what decision theories get the best results (as you can see by observing the offshoot of this conversation between Wedrifid and myself,) but what we do generally agree on here is that the quality of decision theories is determined by their results. If you argue yourself into a decision theory that doesn’t serve you well, you’ve only managed to shoot yourself in the foot.
In the absence of my decision affecting the boxes, taking one box and leaving $1000 on the table still looks like shooting myself in the foot. (Of course if I had the ability to precommit to one-box I would—so, okay, if Omega ever asks me this I will take one box. But if Omega asked me to make a decision after filling the boxes and before I’d made a precommitment… still two boxes.)
I think I’m going to back out of this discussion until I understand decision theory a bit better.
Feel free. You can revisit this conversation any time you feel like it. Discussion threads never really die here, there’s no community norm against replying to comments long after they’re posted.