There are a couple of things I find odd about this. First, it seems to be taken for granted that one-boxing is obviously better than two boxing, but I’m not sure that’s right. J.M. Joyce has an argument (in his foundations of causal decision theory) that is supposed to convince you that two-boxing is the right solution. Importantly, he accepts that you might still wish you weren’t a CDT (so that Omega predicted you would one-box). But, he says, in either case, once the boxes are in front of you, whether you are a CDT or a EDT, you should two-box! The dominance reasoning works in either case, once the prediction has been made and the boxes are in front of you.
But this leads me on to my second point. I’m not sure how much of a flaw Newcomb’s problem is in a decision theory, given that it relies on the intervention of an alien that can accurately predict what you will do. Let’s leave aside the general problem of predicting real agents’ actions with that degree of accuracy. If you know that the prediction of your choice affects the success of your choices, I think that reflexivity or self reference simply makes the prediction meaningless. We’re all used to self-reference being tricky, and I think in this case it just undermines the whole set up. That is, I don’t see the force of the objection from Newcomb’s problem, because I don’t think it’s a problem we could ever possibly face.
Here’s an example of a related kind of “reflexivity makes prediction meaningless”. Let’s say Omega bets you $100 that she can predict what you will eat for breakfast. Once you accept this bet, you now try to think of something that you would never otherwise think to eat for breakfast, in order to win the bet. The fact that your actions and the prediction of your actions have been connected in this way by the bet makes your actions unpredictable.
Going on to the prisoner’s dilemma. Again, I don’t think that it’s the job of decision theory to get “the right” result in PD. Again, the dominance reasoning seems impeccable to me. In fact, I’m tempted to say that I would want any future advanced decision theory to satisfy some form of this dominance principle: it’s crazy to ever choice an act that is guaranteed to be worse. All you need to do to “fix” PD is to have the agent attach enough weight to the welfare of others. That’s not a modification of the decision theory, that’s a modification of the utility function.
But as I understand it, proponents of alternative DTs are talking about a conditional PD where you know you face an opponent executing a particular DT. The fancy-DT-users all defect on PD when the prior of their PD-partner being on CDT or similar is high enough, right?
Wouldn’t you like to be the type of agent who cooperates with near-copies of yourself? Wouldn’t you like to be the type of agent who one-boxes? The trick is to satisfy this desire without using a bunch of stupid special-case rules, and show that it doesn’t lead to poor decisions elsewhere.
But as I understand it, proponents of alternative DTs are talking about a conditional PD where you know you face an opponent executing a particular DT. The fancy-DT-users all defect on PD when the prior of their PD-partner being on CDT or similar is high enough, right?
Wouldn’t you like to be the type of agent who cooperates with near-copies of yourself? Wouldn’t you like to be the type of agent who one-boxes?
Yes, but it would be strictly better (for me) to be the kind of agent who defects against near-copies of myself when they co-operate in one-shot games. It would be better to be the kind of agent who is predicted to one-box, but then two-box once the money has been put in the opaque box.
But the point is really that I don’t see it as the job of an alternative decision theory to get “the right” answers to these sorts of questions.
They’re not necessarily impossible. If you have genuine reason to believe you can outsmart Omega, or that you can outsmart the near-copy of yourself in PD, then you should two-box or defect.
But if the only information you have is that you’re playing against a near-copy of yourself in PD, then cooperating is probably the smart thing to do. I understand this kind of thing is still being figured out.
According to what rules? And anyway I have preferences for all kinds of impossible things. For example, I prefer cooperating with copies of myself, even though I know it would never happen, since we’d both accept the dominance reasoning and defect.
I think he meant according to the rules of the thought experiments. In Newcomb’s problem, Omega predicts what you do. Whatever you choose to do, that’s what Omega predicted you would choose to do. You cannot to choose to do something that Omega wouldn’t predict—it’s impossible. There is no such thing as “the kind of agent who is predicted to one-box, but then two-box once the money has been put in the opaque box”.
Right. The rules of the respective thought experiments.. Similarly, if you’re the sort to defect against near copies of yourself in one-shot PD, then so is your near copy. (edit: I see now that scmbradley already wrote about that—sorry for the redundancy).
Here’s an example of a related kind of “reflexivity makes prediction meaningless”. Let’s say Omega bets you $100 that she can predict what you will eat for breakfast. Once you accept this bet, you now try to think of something that you would never otherwise think to eat for breakfast, in order to win the bet. The fact that your actions and the prediction of your actions have been connected in this way by the bet makes your actions unpredictable.
Your actions have been determined in part by the bet that Omega has made with you—I do not see how that is supposed to make them unpredictable any more than adding any other variable would do so. Remember: You only appear to have free will from within the algorithm, you may decide to think of something you’d never otherwise think about but Omega is advanced enough to model you down to the most basic level—it can predict your more complex behaviours based upon the combination of far simpler rules. You cannot necessarily just decide to think of something random which would be required in order to be unpredictable.
Similarly, the whole question of whether you should choose to two box or one box is a bit iffy. Strictly speaking there’s no SHOULD about it. You will one box or you will two box. The question phrased as a should question—as a choice—is meaningless unless you’re treating choice as a high-level abstraction of lower level rules; and if you do that, then the difficult disappears—just as you don’t ask a rock whether it should or shouldn’t crush someone when it falls down a hill.
Meaningfully, we might ask whether it is preferable to be the type of person who two boxes or the type of person who one boxes. As it turns out it seems to be more preferable to one-box and make stinking great piles of dosh. And as it turns out I’m the sort of person who, holding a desire for filthy lucre, will do so.
It’s really difficult to side step your intuitions—your illusion that you actually get a free choice here. And I think the phrasing of the problem and its answers themselves have a lot to do with that. I think if you think that people get a choice—and the mechanisms of Omega’s prediction hinge upon you being strongly determined—then the question just ceases to make sense. And you’ve got to jettison one of the two; either Omega’s prediction ability or your ability to make a choice in the sense conventionally meant.
we might ask whether it is preferable to be the type of person who two boxes or the type of person who one boxes. As it turns out it seems to be more preferable to one-box
No. What is preferable is to be the kind of person Omega will predict will one-box, and then actually two-box. As long as you “trick” Omega, you get strictly more money. But I guess your point is you can’t trick Omega this way.
Which brings me back to whether Omega is feasible. I just don’t share the intuition that Omega is capable of the sort of predictive capacity required of it.
Which brings me back to whether Omega is feasible. I just don’t share the intuition that Omega is capable of the sort of predictive capacity required of it.
Well, I guess my response to that would be that it’s a thought experiment. Omega’s really just an extreme—hypothetical—case of a powerful predictor, that makes problems in CDT more easily seen by amplifying them. If we were to talk about the prisoner’s dilemma, we could easily have roughly the same underlying discussion.
See mine and orthonormal’s comments on the PD on this post for my view of that.
The point I’m struggling to express is that I don’t think we should worry about the thought experiment, because I have the feeling that Omega is somehow impossible. The suggestion is that Newcomb’s problem makes a problem with CDT clearer. But I argue that Newcomb’s problem makes the problem. The flaw is not with the decision theory, but with the concept of such a predictor. So you can’t use CDT’s “failure” in this circumstance as evidence that CDT is wrong.
Here’s a related point: Omega will never put the money in the box. Smith act like a one-boxer. Omega predicts that Smith will one-box. So the million is put in the opaque box. Now Omega reasons as follows: “Wait though. Even if Smith is a one-boxer, now that I’ve fixed what will be in the boxes, Smith is better off two-boxing. Smith is smart enough to realise that two-boxing is dominant, once I can’t causally affect the contents of the boxes.” So Omega doesn’t put the money in the box.
Would one-boxing ever be advantageous if Omega were reasoning like that? No. The point is Omega will always reason that two-boxing dominates once the contents are fixed. There seems to be something unstable about Omega’s reasoning. I think this is related to why I feel Omega is impossible. (Though I’m not sure how the points interact exactly.)
Here’s a related point: Omega will never put the money in the box. Smith act like a one-boxer. Omega predicts that Smith will one-box. So the million is put in the opaque box. Now Omega reasons as follows: “Wait though. Even if Smith is a one-boxer, now that I’ve fixed what will be in the boxes, Smith is better off two-boxing. Smith is smart enough to realise that two-boxing is dominant, once I can’t causally affect the contents of the boxes.” So Omega doesn’t put the money in the box.
By that logic, you can never win in Kavka’s toxin/Parfit’s hitchhiker scenario.
So I agree. It’s lucky I’ve never met a game theorist in the desert.
Less flippantly. The logic pretty much the same yes. But I don’t see that as a problem for the point I’m making; which is that the perfect predictor isn’t a thought experiment we should worry about.
“Wait though. Even if Smith is a one-boxer, now that I’ve fixed what will be in the boxes, Smith is better off two-boxing. Smith is smart enough to realise that two-boxing is dominant, once I can’t causally affect the contents of the boxes.” So Omega doesn’t put the money in the box.
That line of reasoning is though available to Smith as well, so he can choose to one-boxing because he knows that Omega is a perfect predictor. You’re right to say that the interplay between Omega-prediction-of-Smith and Smith-prediction-of-Omega are in a meta-stable state, BUT: Smith has to decide, he is going to make a decision, and so whatever algorithm it implements, if it ever goes down this line of meta-stable reasoning, must have a way to get out and choose something, even if it’s just bounded computational power (or the limit step of computation in Hamkins infinite Turing machine). But since Omega is a perfect predictor, it will know that and choose accordingly.
I have the feeling that Omega existence is something like an axiom, you can refuse or accept it and both stances are coherent.
Well, i can implement omega by scanning your brain and simulating you. The other ‘non implementations’ of omega, though, imo are best ignored entirely. You can’t really blame a decision theory for failure if there’s no sensible model of the world for it to use.
My decision theory, personally, allows me to ignore unknown and edit my expected utility formula in ad-hoc way if i’m sufficiently convinced that omega will work as described. I think that’s practically useful because effective heuristics often have to be invented on spot without sufficient model of the world.
edit: albeit, if i was convinced that omega works as described, i’d be convinced that it has scanned my brain and is emulating my decision procedure, or is using time travel, or is deciding randomly then destroying the universes where it was wrong… with more time i can probably come up with other implementations, the common thing about the implementations though is that i should 1-box.
People with memory problems tend to repeat “spontaneous” interactions in essentially the same way, which is evidence that quantum noise doesn’t usually sway choices.
You cannot necessarily just decide to think of something random which would be required in order to be unpredictable.
Presented with this scenario, I’d come up with a scheme describing a table of as many different options as I could manage—ideally a very large number, but the combinatorics would probably get unwieldy after a while—and pull numbers from http://www.fourmilab.ch/hotbits/ to make a selection. I might still lose, but knowing (to some small p-value) that it’s possible to predict radioactive decay would easily be worth $100.
Again, the dominance reasoning seems impeccable to me. In fact, I’m tempted to say that I would want any future advanced decision theory to satisfy some form of this dominance principle: it’s crazy to ever choice an act that is guaranteed to be worse.
It’s not always cooperating- that would be dumb. The claim is that there can be improvements on what a CDT algorithm can achieve: TDT or UDT still defects against an opponent that always defects or always cooperates, but achieves (C,C) in some situations where CDT gets (D,D). The dominance reasoning is only impeccable if agents’ decisions really are independent, just like certain theorems in probability only hold when the random variables are independent. (And yes, this is a precisely analogous meaning of “independent”.)
Aha. So when agents’ actions are probabilistically independent, only then does the dominance reasoning kick in?
So the causal decision theorist will say that the dominance reasoning is applicable whenever the agents’ actions are causally independent. So do these other decision theories deny this? That is, do they claim that the dominance reasoning can be unsound even when my choice doesn’t causally impact the choice of the other?
That’s one valid way of looking at the distinction.
CDT allows the causal link from its current move in chess to its opponent’s next move, so it doesn’t view the two as independent.
In Newcomb’s Problem, traditional CDT doesn’t allow a causal link from its decision now to Omega’s action before, so it applies the independence assumption to conclude that two-boxing is the dominant strategy. Ditto with playing PD against its clone.
(Come to think of it, it’s basically a Markov chain formalism.)
So these alternative decision theories have relations of dependence going back in time? Are they sort of couterfactual dependences like “If I were to one-box, Omega would have put the million in the box”? That just sounds like the Evidentialist “news value” account. So it must be some other kind of relation of dependence going backwards in time that rules out the dominance reasoning. I guess I need “Other Decision Theories: A Less Wrong Primer”.
(gah. I wanted to delete this because I decided it was sort of a useless thing to say, but now it’s here in distracting retracted form, being even worse)
All you need to do to “fix” PD is to have the agent attach enough weight to the welfare of others. That’s not a modification of the decision theory, that’s a modification of the utility function.
And it’s arguably telling that this is the solution evolution found. Humans are actually pretty good at avoiding proper prisoners’ dilemmas, due to our somewhat pro-social utility functions.
There are a couple of things I find odd about this. First, it seems to be taken for granted that one-boxing is obviously better than two boxing, but I’m not sure that’s right. J.M. Joyce has an argument (in his foundations of causal decision theory) that is supposed to convince you that two-boxing is the right solution. Importantly, he accepts that you might still wish you weren’t a CDT (so that Omega predicted you would one-box). But, he says, in either case, once the boxes are in front of you, whether you are a CDT or a EDT, you should two-box! The dominance reasoning works in either case, once the prediction has been made and the boxes are in front of you.
But this leads me on to my second point. I’m not sure how much of a flaw Newcomb’s problem is in a decision theory, given that it relies on the intervention of an alien that can accurately predict what you will do. Let’s leave aside the general problem of predicting real agents’ actions with that degree of accuracy. If you know that the prediction of your choice affects the success of your choices, I think that reflexivity or self reference simply makes the prediction meaningless. We’re all used to self-reference being tricky, and I think in this case it just undermines the whole set up. That is, I don’t see the force of the objection from Newcomb’s problem, because I don’t think it’s a problem we could ever possibly face.
Here’s an example of a related kind of “reflexivity makes prediction meaningless”. Let’s say Omega bets you $100 that she can predict what you will eat for breakfast. Once you accept this bet, you now try to think of something that you would never otherwise think to eat for breakfast, in order to win the bet. The fact that your actions and the prediction of your actions have been connected in this way by the bet makes your actions unpredictable.
Going on to the prisoner’s dilemma. Again, I don’t think that it’s the job of decision theory to get “the right” result in PD. Again, the dominance reasoning seems impeccable to me. In fact, I’m tempted to say that I would want any future advanced decision theory to satisfy some form of this dominance principle: it’s crazy to ever choice an act that is guaranteed to be worse. All you need to do to “fix” PD is to have the agent attach enough weight to the welfare of others. That’s not a modification of the decision theory, that’s a modification of the utility function.
I generally share your reservations.
But as I understand it, proponents of alternative DTs are talking about a conditional PD where you know you face an opponent executing a particular DT. The fancy-DT-users all defect on PD when the prior of their PD-partner being on CDT or similar is high enough, right?
Wouldn’t you like to be the type of agent who cooperates with near-copies of yourself? Wouldn’t you like to be the type of agent who one-boxes? The trick is to satisfy this desire without using a bunch of stupid special-case rules, and show that it doesn’t lead to poor decisions elsewhere.
(Yes, you are correct!)
Yes, but it would be strictly better (for me) to be the kind of agent who defects against near-copies of myself when they co-operate in one-shot games. It would be better to be the kind of agent who is predicted to one-box, but then two-box once the money has been put in the opaque box.
But the point is really that I don’t see it as the job of an alternative decision theory to get “the right” answers to these sorts of questions.
The larger point makes sense. Those two things you prefer are impossible according to the rules, though.
They’re not necessarily impossible. If you have genuine reason to believe you can outsmart Omega, or that you can outsmart the near-copy of yourself in PD, then you should two-box or defect.
But if the only information you have is that you’re playing against a near-copy of yourself in PD, then cooperating is probably the smart thing to do. I understand this kind of thing is still being figured out.
According to what rules? And anyway I have preferences for all kinds of impossible things. For example, I prefer cooperating with copies of myself, even though I know it would never happen, since we’d both accept the dominance reasoning and defect.
I think he meant according to the rules of the thought experiments. In Newcomb’s problem, Omega predicts what you do. Whatever you choose to do, that’s what Omega predicted you would choose to do. You cannot to choose to do something that Omega wouldn’t predict—it’s impossible. There is no such thing as “the kind of agent who is predicted to one-box, but then two-box once the money has been put in the opaque box”.
Elsewhere on this comment thread I’ve discussed why I think those “rules” are not interesting. Basically, because they’re impossible to implement.
Right. The rules of the respective thought experiments.. Similarly, if you’re the sort to defect against near copies of yourself in one-shot PD, then so is your near copy. (edit: I see now that scmbradley already wrote about that—sorry for the redundancy).
Your actions have been determined in part by the bet that Omega has made with you—I do not see how that is supposed to make them unpredictable any more than adding any other variable would do so. Remember: You only appear to have free will from within the algorithm, you may decide to think of something you’d never otherwise think about but Omega is advanced enough to model you down to the most basic level—it can predict your more complex behaviours based upon the combination of far simpler rules. You cannot necessarily just decide to think of something random which would be required in order to be unpredictable.
Similarly, the whole question of whether you should choose to two box or one box is a bit iffy. Strictly speaking there’s no SHOULD about it. You will one box or you will two box. The question phrased as a should question—as a choice—is meaningless unless you’re treating choice as a high-level abstraction of lower level rules; and if you do that, then the difficult disappears—just as you don’t ask a rock whether it should or shouldn’t crush someone when it falls down a hill.
Meaningfully, we might ask whether it is preferable to be the type of person who two boxes or the type of person who one boxes. As it turns out it seems to be more preferable to one-box and make stinking great piles of dosh. And as it turns out I’m the sort of person who, holding a desire for filthy lucre, will do so.
It’s really difficult to side step your intuitions—your illusion that you actually get a free choice here. And I think the phrasing of the problem and its answers themselves have a lot to do with that. I think if you think that people get a choice—and the mechanisms of Omega’s prediction hinge upon you being strongly determined—then the question just ceases to make sense. And you’ve got to jettison one of the two; either Omega’s prediction ability or your ability to make a choice in the sense conventionally meant.
No. What is preferable is to be the kind of person Omega will predict will one-box, and then actually two-box. As long as you “trick” Omega, you get strictly more money. But I guess your point is you can’t trick Omega this way.
Which brings me back to whether Omega is feasible. I just don’t share the intuition that Omega is capable of the sort of predictive capacity required of it.
Well, I guess my response to that would be that it’s a thought experiment. Omega’s really just an extreme—hypothetical—case of a powerful predictor, that makes problems in CDT more easily seen by amplifying them. If we were to talk about the prisoner’s dilemma, we could easily have roughly the same underlying discussion.
See mine and orthonormal’s comments on the PD on this post for my view of that.
The point I’m struggling to express is that I don’t think we should worry about the thought experiment, because I have the feeling that Omega is somehow impossible. The suggestion is that Newcomb’s problem makes a problem with CDT clearer. But I argue that Newcomb’s problem makes the problem. The flaw is not with the decision theory, but with the concept of such a predictor. So you can’t use CDT’s “failure” in this circumstance as evidence that CDT is wrong.
Here’s a related point: Omega will never put the money in the box. Smith act like a one-boxer. Omega predicts that Smith will one-box. So the million is put in the opaque box. Now Omega reasons as follows: “Wait though. Even if Smith is a one-boxer, now that I’ve fixed what will be in the boxes, Smith is better off two-boxing. Smith is smart enough to realise that two-boxing is dominant, once I can’t causally affect the contents of the boxes.” So Omega doesn’t put the money in the box.
Would one-boxing ever be advantageous if Omega were reasoning like that? No. The point is Omega will always reason that two-boxing dominates once the contents are fixed. There seems to be something unstable about Omega’s reasoning. I think this is related to why I feel Omega is impossible. (Though I’m not sure how the points interact exactly.)
By that logic, you can never win in Kavka’s toxin/Parfit’s hitchhiker scenario.
So I agree. It’s lucky I’ve never met a game theorist in the desert.
Less flippantly. The logic pretty much the same yes. But I don’t see that as a problem for the point I’m making; which is that the perfect predictor isn’t a thought experiment we should worry about.
That line of reasoning is though available to Smith as well, so he can choose to one-boxing because he knows that Omega is a perfect predictor. You’re right to say that the interplay between Omega-prediction-of-Smith and Smith-prediction-of-Omega are in a meta-stable state, BUT: Smith has to decide, he is going to make a decision, and so whatever algorithm it implements, if it ever goes down this line of meta-stable reasoning, must have a way to get out and choose something, even if it’s just bounded computational power (or the limit step of computation in Hamkins infinite Turing machine). But since Omega is a perfect predictor, it will know that and choose accordingly. I have the feeling that Omega existence is something like an axiom, you can refuse or accept it and both stances are coherent.
Well, i can implement omega by scanning your brain and simulating you. The other ‘non implementations’ of omega, though, imo are best ignored entirely. You can’t really blame a decision theory for failure if there’s no sensible model of the world for it to use.
My decision theory, personally, allows me to ignore unknown and edit my expected utility formula in ad-hoc way if i’m sufficiently convinced that omega will work as described. I think that’s practically useful because effective heuristics often have to be invented on spot without sufficient model of the world.
edit: albeit, if i was convinced that omega works as described, i’d be convinced that it has scanned my brain and is emulating my decision procedure, or is using time travel, or is deciding randomly then destroying the universes where it was wrong… with more time i can probably come up with other implementations, the common thing about the implementations though is that i should 1-box.
Provided my brain’s choice isn’t affected by quantum noise, otherwise I don’t think you can. :-)
People with memory problems tend to repeat “spontaneous” interactions in essentially the same way, which is evidence that quantum noise doesn’t usually sway choices.
Good point. Still, the brain’s choice can be quite deterministic, if you give it enough thought—averaging out noise.
Presented with this scenario, I’d come up with a scheme describing a table of as many different options as I could manage—ideally a very large number, but the combinatorics would probably get unwieldy after a while—and pull numbers from http://www.fourmilab.ch/hotbits/ to make a selection. I might still lose, but knowing (to some small p-value) that it’s possible to predict radioactive decay would easily be worth $100.
Of course, that’s the smartassed answer.
Well the smartarse response is that Omega’s just plugged himself in on the other end of your hotbits request =p
It’s not always cooperating- that would be dumb. The claim is that there can be improvements on what a CDT algorithm can achieve: TDT or UDT still defects against an opponent that always defects or always cooperates, but achieves (C,C) in some situations where CDT gets (D,D). The dominance reasoning is only impeccable if agents’ decisions really are independent, just like certain theorems in probability only hold when the random variables are independent. (And yes, this is a precisely analogous meaning of “independent”.)
Aha. So when agents’ actions are probabilistically independent, only then does the dominance reasoning kick in?
So the causal decision theorist will say that the dominance reasoning is applicable whenever the agents’ actions are causally independent. So do these other decision theories deny this? That is, do they claim that the dominance reasoning can be unsound even when my choice doesn’t causally impact the choice of the other?
That’s one valid way of looking at the distinction.
CDT allows the causal link from its current move in chess to its opponent’s next move, so it doesn’t view the two as independent.
In Newcomb’s Problem, traditional CDT doesn’t allow a causal link from its decision now to Omega’s action before, so it applies the independence assumption to conclude that two-boxing is the dominant strategy. Ditto with playing PD against its clone.
(Come to think of it, it’s basically a Markov chain formalism.)
So these alternative decision theories have relations of dependence going back in time? Are they sort of couterfactual dependences like “If I were to one-box, Omega would have put the million in the box”? That just sounds like the Evidentialist “news value” account. So it must be some other kind of relation of dependence going backwards in time that rules out the dominance reasoning. I guess I need “Other Decision Theories: A Less Wrong Primer”.
(gah. I wanted to delete this because I decided it was sort of a useless thing to say, but now it’s here in distracting retracted form, being even worse)
And it’s arguably telling that this is the solution evolution found. Humans are actually pretty good at avoiding proper prisoners’ dilemmas, due to our somewhat pro-social utility functions.