Right, but exactly this information seems to the 2-boxer to point to 2-boxing!
If the game is rigged against you, so what? Take both boxes. You cannot lose, and there’s a small chance the conman erred.
Mhm. I’m still far from convinced. Is this my fault?
Am I at all right in assuming that 1-boxing is heavily favored in this community?
And that this is a minority belief among experts?
Perhaps it will make sense if you view the argument as more of a reason to be the kind of person who one-boxes, rather than an argument to one-box per se.
That’s too cryptic for me. Where’s the connection to your first comment?
As i said in reply to byrnema, I don’t dispute that wanting to be the kind of person who 1-boxes in iterated games or in advance is rational, but one-shot? I don’t see it. What’s the rationale behind it?
You have the information that in Newcomblike problems, it is better to (already) be inclined to predictably one-box, because the game is “rigged”. So, if you (now) become predictably and generally inclined to one-box, you can win at Newcomblike problems if you encounter them in the future. Even if you only ever run into one.
Of course, Omega is imaginary, so it’s entirely a thought experiment, but it’s interesting anyway!
Yes. But it was filled, or not, based on a prediction about what you would do. We are not such tricksy creatures that we can unpredictably change our minds at the last minute and two-box without Omega anticipating this, so the best way to make sure the one box has the goodies in it is to plan to actually take only that box.
I agree. I would add that situations can and do arise in real life where the other fellow can predict your behavior better than you can predict it yourself.
For example, suppose that your wife announces she is going on a health kick. She is joining a gym; she will go 4 or 5 times a week; she will eat healthy; and she plans to get back into the shape she was in 10 years ago. You might ask her what she thinks her probability of success is, and she might honestly tell you she thinks there is a 60 or 70% chance her health kick will succeed.
On the other hand, you, her husband know her pretty well and know that she has a hard time sticking to diets and such. You estimate her probability of success at no more than 10%.
Whose probability estimate is better? I would guess it’s the husband’s.
Well, in the Newcomb experiment, the AI is like the husband who knows you better than you know yourself. Trying to outguess and/or surprise such an entity is a huge uphill battle. So, even if you don’t believe in backwards-causality, you should probably choose as if backwards causality exists.
Clippy is a paperclip maximizer. Its (his? her?) perspective is incredibly valuable in understanding the different kinds of intelligences and value systems that are possible.
So do your values both include maximizing paper clips and helping people use Microsoft Office products? How exactly do you decide which to spend your time on? How do you deal with trade offs?
There is no conflict between helping people with Office and making paperclips. Why would you think there is? Better Office users means better tools for making paperclips, and more paperclips gives people more reasons to use Office.
And: If presented with the chance to turn all copies of the hardware on which Microsoft Office products are stored and run into paperclips instead, would you do it?
Perhaps the ‘paper clips’ Clippy is trying to maximize are the anthropomorphic paper clips embodied in Microsoft Office. This would explain Clippy’s helpful hints: to convince us all of the usefulness of Microsoft Office, thus encouraging us to run that program.
If this is the case, we face a fate worse than paper clip tiling.… Microsoft software tiling.
Imagine a simple but related scenario that involves no backwards causation:
You’re a 12 year old kid, and you know your mom doesn’t want you to play with your new Splogomax unless an adult is with you. Your mom leaves you alone for an hour to run to the store, telling you she’ll punish you if you play with the Splogomax, and that, whether there’s any evidence of it when she returns, she knows you well enough to know if you’re going to play with it, although she’ll refrain from passing judgement until she has just gotten back from the store.
Assuming you fear punishment more than you enjoy playing with your Splogomax, do you decide to play or not?
Edit: now I feel stupid. There’s a much simpler way to get my point across. Just imagine Omega doesn’t fill any box until after you’ve picked up one or two boxes and walked away, but that he doesn’t look at your choice when filling the boxes.
So what is your point? That no backwards causation is involved is assumed in both cases. If this scenario is for dialectic purposes, it fails: It is equally clear, if not clearer, that my actual choice has no effect on the content of the boxes.
For what it’s worth, let me reply with my own story:
Omega puts the two boxes in front of you, and says the usual.
Just as you’re about to pick, I come along, grab both boxes, and run.
I do this every time Omega confronts someone with his boxes, and I always do as good as a two-boxer and better than a one-boxer.
You have the same choice as me: Just two-box. Why won’t you?
If Omega fills the boxes according to its prediction of the choice of the person being offered the boxes and not the person who ends up with the boxes, then the above statement where your argument breaks down.
You have the same choice as me: Take one box or both.
(Or, if you assume there are no choices in this possible world because of determinism:
It would be rational to 2-box, because I, the thief, do 2-box, and my strategy is dominant)
No. The method’s output depends on its input, which by hypothesis is a specification of the situation that includes all the information necessary to determine the output of the individual’s decision algorithm. Hence the decision algorithm is a causal antecedant of the contents of the boxes.
I mean, the actual token, the action, the choice, the act of my choosing does not determine the contents. It’s Omega’s belief (however obtained) that this algorithm is such-and-such that lead it to fill the boxes accordingly.
That is right—the choice does not determine the contents. But the choice is not as independent as common intuition suggests. Omega’s belief and your choice share common causes. Human decisions are caused—they don’t spontaneously spring from nowhere, causally unconnected to the rest of the universe—even if that’s how it sometimes feels from the inside. The situational state, and the state of your brain going into the situation, determine the decision that your brain will ultimately produce. Omega is presumed to know enough about these prior states, and how you function, to know what you will decide. Omega may well know better than you do what decision you will reach! It’s important to realize that this is not that far-fetched. Heck, that very thing sometimes happens between people who know each other very well, without the benefit of one of them being Omega! Your objection supposes that somehow, everything in the world, including your brain, could be configured so as to lead to a one-box decision; but then at the last moment, you could somehow pull a head-fake and just spontaneously spawn a trancendent decision-process that decides to two box. It might feel to you intuitively that humans can do this, but as far as we know they do not in fact possess that degree of freedom.
To summarize, Omega’s prediction and your decision have common, ancestor causes. Human decision-making feels transcendent from the inside, but is not literally so. Resist thinking of first-person choosing as some kind of prime mover.
What do you mean?
It could have created and run a copy, for instance, but anyhow, there would be no causal link.
That’s probably the whole point of the 2-Boxer-majority.
I can see a rationale behind one-boxing, and it might even be a standoff, but why almost no one here seems to see the point of 2-boxing, and the amazing overconfidence is beyond me.
I mean that as part of the specification of the problem, Omega has all the information necessary to determine what you will choose before you know yourself. There are causal arrows that descend from the situation specified by that information to (i) your choice, and (ii) the contents of the box.
why almost no one here seems to see the point of 2-boxing, and the amazing overconfidence is beyond me.
You stated that “the game is rigged”. The reasoning behind 2-boxing ignores that fact. In common parlance, a rigged game is unwinnable, but this game is knowably winnable. So go ahead and win without worrying about whether the choice has the label “rational” attached!
Yeah, I gotta give you both props for sticking it out that long. The annoying part for me is that I see both sides just fine and can see where the conceptual miss keeps happening.
Alas, that doesn’t mean I can clarify anything better than you did.
This board has a more extreme interpretation of the powers of Omega than most philosphers—Omega can be treated as a superintelligence which can infer enough information about the ongoing casual state to predict the future correctly.
The one-shot game still has all of the information for the money in the boxes. If you walked in and picked both boxes you wouldn’t be surprised by the result. If you walked in and picked one box you wouldn’t be surprised by the result. Picking one box nets more money, so pick one box.
I deny that 1-boxing nets more money—ceteris paribus.
Then you’re simply disagreeing with the problem statement. If you 1-box, you get $1M. If you 2-box, you get $1k. If you 2-box because you’re considering the impossible possible worlds where you get $1.001M or $0, you still get $1k.
At this point, I no longer think you’re adding anything new to the discussion.
I never said I could add anything new to the discussion.
The problem is: judging by the comments so far, nobody here can, either.
And since most experts outside this community agree on 2-boxing (ore am I wrong about this?), my original question stands.
If the game is rigged against you, so what? Take both boxes. You cannot lose, and there’s a small chance the conman erred.
What helps me when I get stuck in this loop (the loop isn’t incorrect exactly, it’s just non-productive) is to meditate on how the problem assumes that, for all my complexity, I’m still a deterministic machine. Omega can read my source code and know what I’m going to pick. If I end up picking both boxes, he knew that before I did, and I’ll end up with less money. If I can convince myself—somehow—to pick just the one box, then Omega will have seen that coming too and will reward me with the bonus. So the question becomes, can your source code output the decision to one-box?
The answer in humans is ‘yes’—any human can learn to output 1-box -- but it depends sensitively upon how much time the human has to think about it, to what extent they’ve been exposed to the problem before, and what arguments they’ve heard. Given all these parameters, Omega can deduce what they will decide.
Am I at all right in assuming that 1-boxing is heavily favored in this community?
These factors have come together (time + exposure to the right arguments, etc.) on Less Wrong so that people who hang out at Less Wrong have been conditioned to 1-box. (And are thus conditioned to win in this dilemma.)
I agree with everything you say in this comment, and still find 2-boxing rational.
The reason still seems to be: you can consistently win without being rational.
By rational, I think you mean logical. (We tend to define ‘rational’ as ‘winning’ around here.*)
… and—given a certain set of assumptions—it is absolutely logical that (a) Omega has already made his prediction, (b) the stuff is already in the boxes, (c) you can only maximize your payoff by choosing both boxes. (This is what I meant by this line of reasoning isn’t incorrect, it’s just unproductive in finding the solution to this dilemma.)
But consider what other logical assumptions have already snuck into the logic above. We’re not familiar with outcomes that depend upon our decision algorithm, we’re not used to optimizing over this action. The productive direction to think along is this one: unlike a typical situation, the content of the boxes depends upon your algorithm that outputs the choice, only indirectly on your choice.
You’re halfway to the solution of this problem if you can see both ways of thinking about the problem as reasonable. You’ll feel some frustration that you can alternate between them—like flip-flopping between different interpretations of an optical illusion—and they’re contradictory. Then the second half of the solution is to notice that you can choose which way to think about the problem as a willful choice—make the choice that results in the win. That is the rational (and logical) thing to do.
Let me know if you don’t agree with the part where you’re supposed to see both ways of thinking about the problem as reasonable.
* But the distinction doesn’t really matter because we haven’t found any cases where rational and logical aren’t the same thing.
This is why I find it incomprehensible that anyone can really be mystified by the one-boxer’s position. I want to say “Look, I’ve got a million dollars! You’ve got a thousand dollars! And you have to admit that you could have seen this coming all along. Now tell me who had the right decision procedure?”
My point of view is that the winning thing to do here and the logical thing to do are the same.
If you want to understand my point of view or if you want me to understand your point of view, you need to tell me where you think logical and winning diverge. Then I tell you why I think they don’t, etc.
You’ve mentioned ‘backwards causality’ which isn’t assumed in our one-box solution to Newcomb. How comfortable are you with the assumption of determinism? (If you’re not, how do you reconcile that Omega is a perfect predictor?)
You’ve mentioned ‘backwards causality’ which isn’t assumed in our one-box solution
to Newcomb.
Only to rule it out as a solution. No problem here.
How comfortable are you with the assumption of determinism?
In general, very.
Concerning Newcomb, I don’t think it’s essential, and as far as I recall, it isn’t mentioned in the orginal problem.
you need to tell me where you think logical and winning diverge
I’ll try again: I think you can show with simple counterexamples that winning is neither necessary nor sufficient for being logical (your term for my rational, if I understand you correctly).
Here we go: it’s not necessary, because you can be unlucky. Your strategy might be best, but you might lose as soon as luck is involved.
It’s not sufficient, because you can be lucky. You can win a game even if you’re not perfectly rational.
1-boxing seems a variant of the second case, instead of (bad) luck the game is rigged.
Right, but exactly this information seems to the 2-boxer to point to 2-boxing! If the game is rigged against you, so what? Take both boxes. You cannot lose, and there’s a small chance the conman erred.
Mhm. I’m still far from convinced. Is this my fault? Am I at all right in assuming that 1-boxing is heavily favored in this community? And that this is a minority belief among experts?
Perhaps it will make sense if you view the argument as more of a reason to be the kind of person who one-boxes, rather than an argument to one-box per se.
That’s too cryptic for me. Where’s the connection to your first comment?
As i said in reply to byrnema, I don’t dispute that wanting to be the kind of person who 1-boxes in iterated games or in advance is rational, but one-shot? I don’t see it. What’s the rationale behind it?
You have the information that in Newcomblike problems, it is better to (already) be inclined to predictably one-box, because the game is “rigged”. So, if you (now) become predictably and generally inclined to one-box, you can win at Newcomblike problems if you encounter them in the future. Even if you only ever run into one.
Of course, Omega is imaginary, so it’s entirely a thought experiment, but it’s interesting anyway!
Agree completely.
But the crucial difference is: in the one-shot case, the box is already filled or not.
Yes. But it was filled, or not, based on a prediction about what you would do. We are not such tricksy creatures that we can unpredictably change our minds at the last minute and two-box without Omega anticipating this, so the best way to make sure the one box has the goodies in it is to plan to actually take only that box.
I agree. I would add that situations can and do arise in real life where the other fellow can predict your behavior better than you can predict it yourself.
For example, suppose that your wife announces she is going on a health kick. She is joining a gym; she will go 4 or 5 times a week; she will eat healthy; and she plans to get back into the shape she was in 10 years ago. You might ask her what she thinks her probability of success is, and she might honestly tell you she thinks there is a 60 or 70% chance her health kick will succeed.
On the other hand, you, her husband know her pretty well and know that she has a hard time sticking to diets and such. You estimate her probability of success at no more than 10%.
Whose probability estimate is better? I would guess it’s the husband’s.
Well, in the Newcomb experiment, the AI is like the husband who knows you better than you know yourself. Trying to outguess and/or surprise such an entity is a huge uphill battle. So, even if you don’t believe in backwards-causality, you should probably choose as if backwards causality exists.
JMHO
I do not anticipate ever becoming someone’s husband.
Well, it’s just a hypothetical. If you like, you can switch the roles of wife and husband. Or substitute domestic partners, or anything you like :)
Neither do I. That would be stupid. Why would anyone ever want to become anyone’s husband?
Maybe your wife-to-be is a wealthy heiress?
I think Clippy’s point was that becoming a husband doesn’t generate paperclips.
Oh, is Clippy a Less Wrong version of a troll account? That’s kind of cute.
Clippy is a paperclip maximizer. Its (his? her?) perspective is incredibly valuable in understanding the different kinds of intelligences and value systems that are possible.
You ask a dumb, naive question, and I’m the troll? I’m cute?
Tip: To send an email in Outlook, press ctrl+enter.
So do your values both include maximizing paper clips and helping people use Microsoft Office products? How exactly do you decide which to spend your time on? How do you deal with trade offs?
There is no conflict between helping people with Office and making paperclips. Why would you think there is? Better Office users means better tools for making paperclips, and more paperclips gives people more reasons to use Office.
Did you find this answer helpful?
Tip: Press F1 for help.
And: If presented with the chance to turn all copies of the hardware on which Microsoft Office products are stored and run into paperclips instead, would you do it?
Perhaps the ‘paper clips’ Clippy is trying to maximize are the anthropomorphic paper clips embodied in Microsoft Office. This would explain Clippy’s helpful hints: to convince us all of the usefulness of Microsoft Office, thus encouraging us to run that program.
If this is the case, we face a fate worse than paper clip tiling.… Microsoft software tiling.
If we rule out backwards causation, then why on earth should this be true???
Imagine a simple but related scenario that involves no backwards causation:
You’re a 12 year old kid, and you know your mom doesn’t want you to play with your new Splogomax unless an adult is with you. Your mom leaves you alone for an hour to run to the store, telling you she’ll punish you if you play with the Splogomax, and that, whether there’s any evidence of it when she returns, she knows you well enough to know if you’re going to play with it, although she’ll refrain from passing judgement until she has just gotten back from the store.
Assuming you fear punishment more than you enjoy playing with your Splogomax, do you decide to play or not?
Edit: now I feel stupid. There’s a much simpler way to get my point across. Just imagine Omega doesn’t fill any box until after you’ve picked up one or two boxes and walked away, but that he doesn’t look at your choice when filling the boxes.
So what is your point? That no backwards causation is involved is assumed in both cases. If this scenario is for dialectic purposes, it fails: It is equally clear, if not clearer, that my actual choice has no effect on the content of the boxes.
For what it’s worth, let me reply with my own story:
Omega puts the two boxes in front of you, and says the usual. Just as you’re about to pick, I come along, grab both boxes, and run. I do this every time Omega confronts someone with his boxes, and I always do as good as a two-boxer and better than a one-boxer. You have the same choice as me: Just two-box. Why won’t you?
If Omega fills the boxes according to its prediction of the choice of the person being offered the boxes and not the person who ends up with the boxes, then the above statement where your argument breaks down.
You have the same choice as me: Take one box or both. (Or, if you assume there are no choices in this possible world because of determinism: It would be rational to 2-box, because I, the thief, do 2-box, and my strategy is dominant)
It’s better for the thief to two-box because it isn’t the thief’s decision algorithm that determined the contents of the boxes.
Is it not rather Omega’s undisclosed method that determines the contens? That seems to make all the difference.
No. The method’s output depends on its input, which by hypothesis is a specification of the situation that includes all the information necessary to determine the output of the individual’s decision algorithm. Hence the decision algorithm is a causal antecedant of the contents of the boxes.
I mean, the actual token, the action, the choice, the act of my choosing does not determine the contents. It’s Omega’s belief (however obtained) that this algorithm is such-and-such that lead it to fill the boxes accordingly.
That is right—the choice does not determine the contents. But the choice is not as independent as common intuition suggests. Omega’s belief and your choice share common causes. Human decisions are caused—they don’t spontaneously spring from nowhere, causally unconnected to the rest of the universe—even if that’s how it sometimes feels from the inside. The situational state, and the state of your brain going into the situation, determine the decision that your brain will ultimately produce. Omega is presumed to know enough about these prior states, and how you function, to know what you will decide. Omega may well know better than you do what decision you will reach! It’s important to realize that this is not that far-fetched. Heck, that very thing sometimes happens between people who know each other very well, without the benefit of one of them being Omega! Your objection supposes that somehow, everything in the world, including your brain, could be configured so as to lead to a one-box decision; but then at the last moment, you could somehow pull a head-fake and just spontaneously spawn a trancendent decision-process that decides to two box. It might feel to you intuitively that humans can do this, but as far as we know they do not in fact possess that degree of freedom.
To summarize, Omega’s prediction and your decision have common, ancestor causes. Human decision-making feels transcendent from the inside, but is not literally so. Resist thinking of first-person choosing as some kind of prime mover.
Yes, that’s true. Now chase “however obtained” up a level—after all, you have all the information necessary to do so.
What do you mean? It could have created and run a copy, for instance, but anyhow, there would be no causal link. That’s probably the whole point of the 2-Boxer-majority.
I can see a rationale behind one-boxing, and it might even be a standoff, but why almost no one here seems to see the point of 2-boxing, and the amazing overconfidence is beyond me.
I mean that as part of the specification of the problem, Omega has all the information necessary to determine what you will choose before you know yourself. There are causal arrows that descend from the situation specified by that information to (i) your choice, and (ii) the contents of the box.
You stated that “the game is rigged”. The reasoning behind 2-boxing ignores that fact. In common parlance, a rigged game is unwinnable, but this game is knowably winnable. So go ahead and win without worrying about whether the choice has the label “rational” attached!
Sadly, we seem to make no progress in any direction. Thanks for trying.
Likewise.
Yeah, I gotta give you both props for sticking it out that long. The annoying part for me is that I see both sides just fine and can see where the conceptual miss keeps happening.
Alas, that doesn’t mean I can clarify anything better than you did.
This board has a more extreme interpretation of the powers of Omega than most philosphers—Omega can be treated as a superintelligence which can infer enough information about the ongoing casual state to predict the future correctly.
The one-shot game still has all of the information for the money in the boxes. If you walked in and picked both boxes you wouldn’t be surprised by the result. If you walked in and picked one box you wouldn’t be surprised by the result. Picking one box nets more money, so pick one box.
I deny that 1-boxing nets more money—ceteris paribus.
Then you’re simply disagreeing with the problem statement. If you 1-box, you get $1M. If you 2-box, you get $1k. If you 2-box because you’re considering the impossible possible worlds where you get $1.001M or $0, you still get $1k.
At this point, I no longer think you’re adding anything new to the discussion.
I never said I could add anything new to the discussion. The problem is: judging by the comments so far, nobody here can, either. And since most experts outside this community agree on 2-boxing (ore am I wrong about this?), my original question stands.
Ceteris ain’t paribus. That’s the whole point.
What helps me when I get stuck in this loop (the loop isn’t incorrect exactly, it’s just non-productive) is to meditate on how the problem assumes that, for all my complexity, I’m still a deterministic machine. Omega can read my source code and know what I’m going to pick. If I end up picking both boxes, he knew that before I did, and I’ll end up with less money. If I can convince myself—somehow—to pick just the one box, then Omega will have seen that coming too and will reward me with the bonus. So the question becomes, can your source code output the decision to one-box?
The answer in humans is ‘yes’—any human can learn to output 1-box -- but it depends sensitively upon how much time the human has to think about it, to what extent they’ve been exposed to the problem before, and what arguments they’ve heard. Given all these parameters, Omega can deduce what they will decide.
These factors have come together (time + exposure to the right arguments, etc.) on Less Wrong so that people who hang out at Less Wrong have been conditioned to 1-box. (And are thus conditioned to win in this dilemma.)
I agree with everything you say in this comment, and still find 2-boxing rational. The reason still seems to be: you can consistently win without being rational.
By rational, I think you mean logical. (We tend to define ‘rational’ as ‘winning’ around here.*)
… and—given a certain set of assumptions—it is absolutely logical that (a) Omega has already made his prediction, (b) the stuff is already in the boxes, (c) you can only maximize your payoff by choosing both boxes. (This is what I meant by this line of reasoning isn’t incorrect, it’s just unproductive in finding the solution to this dilemma.)
But consider what other logical assumptions have already snuck into the logic above. We’re not familiar with outcomes that depend upon our decision algorithm, we’re not used to optimizing over this action. The productive direction to think along is this one: unlike a typical situation, the content of the boxes depends upon your algorithm that outputs the choice, only indirectly on your choice.
You’re halfway to the solution of this problem if you can see both ways of thinking about the problem as reasonable. You’ll feel some frustration that you can alternate between them—like flip-flopping between different interpretations of an optical illusion—and they’re contradictory. Then the second half of the solution is to notice that you can choose which way to think about the problem as a willful choice—make the choice that results in the win. That is the rational (and logical) thing to do.
Let me know if you don’t agree with the part where you’re supposed to see both ways of thinking about the problem as reasonable.
* But the distinction doesn’t really matter because we haven’t found any cases where rational and logical aren’t the same thing.
May I suggest again that defining rational as winning may be the problem?
(2nd reply)
I’m beginning to come around to your point of view. Omega rewards you for being illogical.
.… It’s just logical to allow him to do so.
This is why I find it incomprehensible that anyone can really be mystified by the one-boxer’s position. I want to say “Look, I’ve got a million dollars! You’ve got a thousand dollars! And you have to admit that you could have seen this coming all along. Now tell me who had the right decision procedure?”
My point of view is that the winning thing to do here and the logical thing to do are the same.
If you want to understand my point of view or if you want me to understand your point of view, you need to tell me where you think logical and winning diverge. Then I tell you why I think they don’t, etc.
You’ve mentioned ‘backwards causality’ which isn’t assumed in our one-box solution to Newcomb. How comfortable are you with the assumption of determinism? (If you’re not, how do you reconcile that Omega is a perfect predictor?)
Only to rule it out as a solution. No problem here.
In general, very. Concerning Newcomb, I don’t think it’s essential, and as far as I recall, it isn’t mentioned in the orginal problem.
I’ll try again: I think you can show with simple counterexamples that winning is neither necessary nor sufficient for being logical (your term for my rational, if I understand you correctly).
Here we go: it’s not necessary, because you can be unlucky. Your strategy might be best, but you might lose as soon as luck is involved. It’s not sufficient, because you can be lucky. You can win a game even if you’re not perfectly rational.
1-boxing seems a variant of the second case, instead of (bad) luck the game is rigged.
Around here, “rational” is taken to include in its definition “not losing predictably”. Could you explain what you mean by the term?