Basically: EDT/UDT has simple arguments in its favor and seems to perform well. There don’t seem to be any serious arguments in favor of CDT, and the human intuition in its favor seems quite debunkable. So it seems like the burden of proof is on CDT, to justify why it isn’t crazy. It may be that CDT has met that burden, but I’m not aware of it.
A. The dominance arguments in favor of two-boxing seem quite weak. They tend to apply verbatum to playing prisoner’s dilemmas against a mirror (If the mirror cooperates you’d prefer defect, if the mirror defects you’d prefer defect, so regardless of the state of nature you’d prefer defect). So why do you not accept the dominance argument for a mirror, but accept it in the case of Newcomb-like problems? To discriminate the cases it seems you need to make an assumption of no causal connection, or a special role for time, in your argument.
This begs the question terribly—why is a causal connection privileged? Why is the role of time privileged? As far as I can tell these two things are pretty arbitrary and unimportant. I’m not aware of any strong philosophical arguments for CDT, besides “it seems intuitively sensible to a human,” and see below for the debunking of those intuitions. (Again, maybe there are better arguments here, but I’ve never encountered one. Basically I’m looking for any statement of a kind of dominance principle over states of nature, which doesn’t look completely arbitrary and is also at all plausible.)
B. A sophisticated interpretation of EDT (called UDT around here) seems to perform well in all cases we’ve considered, in the sense that an agent making good decisions will achieve good outcomes. I think this is strong evidence in favor of a theory which purports to say which actions are good, since good decisions ought to lead to good outcomes; I agree its not a knock-down argument, but again I know of no serious counterarguments.
C. It seems that EDT is supported by the simplest philosophical arguments. We need to choose between outcomes in which we make decision A vs. decision B. It makes sense to choose between outcomes which we consider to be possible (in which we make decision A or decision B). CDT doesn’t do this, and considers outcomes which are inconsistent with our knowledge of the situation. This isn’t enough to pin down EDT uniquely (though further arguments can), but it does seem like a strong point in favor of EDT over CDT.
D. An agent living in an environment like humans’ will do fine by using CDT, because the only effects of their decisions are causal. CDT is much simpler to run than EDT because it doesn’t rely on a strong self-model (doing EDT without a good self-model results in worse decisions than CDT in reasonable situations; this is basically what the claims that EDT performs badly in such-and-such a situation amount to, at least the ones I have seen). So it seems like we can pretty easily explain why humans have an intuition in favor of CDT, and it seems like extremely weak evidence against EDT/UDT.
I’m happy to learn that you consider UDT a variant of EDT, because after thinking about these issues for awhile my current point of view is that some form of EDT is obviously the correct thing to do, but in standard examples of EDT failing the relevant Bayesian updates are being performed incorrectly. The problem is that forcing yourself into a reference class by performing an action doesn’t make it reasonable for you to reason as if you were a random sample from that reference class, because you aren’t: you introduced a selection bias. Does this agree with your thoughts?
“why is a causal connection privileged?”
I agree with everything here. What follows is merely history.
Historically, I think that CDT was meant to address the obvious shortcomings of choosing to bring about states that were merely correlated with good outcomes (as in the case of whitening one’s teeth to reduce lung cancer risk). When Pearl advocates CDT, he is mainly advocating acting based on robust connections that will survive the perturbation of the system caused by the action itself. (e.g. Don’t think you’ll cure lung cancer by making your population brush their teeth, because that is a non-robust correlation that will be eliminated once you change the system). The centrality of causality in decision making was obvious intuitively but wasn’t reflected in formal Bayesian decision theory. This was because of the lack of a good formalism linking probability and causality (and some erroneous positivistic scruples against the very idea of causality). Pearl and SGS’s work on causality has done much to address this, but I think there is much to be done.
There is a very annoying historical accident where EDT was taken to be the ‘one-boxing’ decision theory. First, any use of probability theory in the NP with infallible predictor is suspicious, because the problem can be specified in a logically complete way with no room for empirical uncertainty. (This is why dominance reasoning is brought in for CDT. What should the probabilities be?). Second, EDT is not easy to make coherent given an agent who knows they follow EDT. (The action that EDT disfavors will have probability zero and so the agent cannot condition on it in traditional probability theory). Third, EDT just barely one-boxes. It doesn’t one-box on Double Transparent Newcomb, nor on Counterfactual Mugging. It’s also obscure what it does on PD. (Again, I can play the PD against a selfish clone of myself, with both agents having each other’s source code. There is no empirical uncertainty here, and so applying probability theory immediate raises deep foundational problems).
If TDT/UDT had come first (including the logical models and deep connections to Godel’s theorem), the philosophy discussion of NP would have been very different. EDT (which brings into the NP very dubious empirical probability distributions) would not have been considered at all for NP. I don’t see that CDT would have held much interest if its alternative was not as feeble as EDT.
It is important to understand why economists have done so much work with Nash Equilibria (e.g. on the PD) rather than invent UDT. This is explained by the fact that the assumption of logical correlation and perfect empirical knowledge between agents in the PD is not the practical reality. This doesn’t mean that UDT is not relevant to practical situations, but only that these situations involve many additional elements that may be complex to deal with in UDT. Causal based theories would have been interesting independently, for the reasons noted above concerning robust correlations.
EDIT: I realize the comment by Paul Christiano sometimes describes UDT as a variant of EDT. When I used the term “EDT” I mean the theory discussed in the philosophy literature which involves choosing the action that maximizes P(outcomes / action). This is a theory which essentially makes use of vanilla conditional probability. In what I say, I assume that UDT/TDT, despite some similarity to EDT in spirit, are not limited to regular conditioning and do not fail on smoking lesion.
I wonder if David Lewis (perhaps the most notorious philosophical two-boxer) was skeptical that any human had a sufficiently strong self-model. I think there are very who few have better self-models than he did, so it’s quite interesting if he did think this. His discussion of the “tickle defence” in his paper “Causal Decision Theory” may point that way.
Basically: EDT/UDT has simple arguments in its favor and seems to perform well. There don’t seem to be any serious arguments in favor of CDT, and the human intuition in its favor seems quite debunkable. So it seems like the burden of proof is on CDT, to justify why it isn’t crazy. It may be that CDT has met that burden, but I’m not aware of it.
A. The dominance arguments in favor of two-boxing seem quite weak. They tend to apply verbatum to playing prisoner’s dilemmas against a mirror (If the mirror cooperates you’d prefer defect, if the mirror defects you’d prefer defect, so regardless of the state of nature you’d prefer defect). So why do you not accept the dominance argument for a mirror, but accept it in the case of Newcomb-like problems? To discriminate the cases it seems you need to make an assumption of no causal connection, or a special role for time, in your argument.
This begs the question terribly—why is a causal connection privileged? Why is the role of time privileged? As far as I can tell these two things are pretty arbitrary and unimportant. I’m not aware of any strong philosophical arguments for CDT, besides “it seems intuitively sensible to a human,” and see below for the debunking of those intuitions. (Again, maybe there are better arguments here, but I’ve never encountered one. Basically I’m looking for any statement of a kind of dominance principle over states of nature, which doesn’t look completely arbitrary and is also at all plausible.)
B. A sophisticated interpretation of EDT (called UDT around here) seems to perform well in all cases we’ve considered, in the sense that an agent making good decisions will achieve good outcomes. I think this is strong evidence in favor of a theory which purports to say which actions are good, since good decisions ought to lead to good outcomes; I agree its not a knock-down argument, but again I know of no serious counterarguments.
C. It seems that EDT is supported by the simplest philosophical arguments. We need to choose between outcomes in which we make decision A vs. decision B. It makes sense to choose between outcomes which we consider to be possible (in which we make decision A or decision B). CDT doesn’t do this, and considers outcomes which are inconsistent with our knowledge of the situation. This isn’t enough to pin down EDT uniquely (though further arguments can), but it does seem like a strong point in favor of EDT over CDT.
D. An agent living in an environment like humans’ will do fine by using CDT, because the only effects of their decisions are causal. CDT is much simpler to run than EDT because it doesn’t rely on a strong self-model (doing EDT without a good self-model results in worse decisions than CDT in reasonable situations; this is basically what the claims that EDT performs badly in such-and-such a situation amount to, at least the ones I have seen). So it seems like we can pretty easily explain why humans have an intuition in favor of CDT, and it seems like extremely weak evidence against EDT/UDT.
I’m happy to learn that you consider UDT a variant of EDT, because after thinking about these issues for awhile my current point of view is that some form of EDT is obviously the correct thing to do, but in standard examples of EDT failing the relevant Bayesian updates are being performed incorrectly. The problem is that forcing yourself into a reference class by performing an action doesn’t make it reasonable for you to reason as if you were a random sample from that reference class, because you aren’t: you introduced a selection bias. Does this agree with your thoughts?
“why is a causal connection privileged?” I agree with everything here. What follows is merely history.
Historically, I think that CDT was meant to address the obvious shortcomings of choosing to bring about states that were merely correlated with good outcomes (as in the case of whitening one’s teeth to reduce lung cancer risk). When Pearl advocates CDT, he is mainly advocating acting based on robust connections that will survive the perturbation of the system caused by the action itself. (e.g. Don’t think you’ll cure lung cancer by making your population brush their teeth, because that is a non-robust correlation that will be eliminated once you change the system). The centrality of causality in decision making was obvious intuitively but wasn’t reflected in formal Bayesian decision theory. This was because of the lack of a good formalism linking probability and causality (and some erroneous positivistic scruples against the very idea of causality). Pearl and SGS’s work on causality has done much to address this, but I think there is much to be done.
There is a very annoying historical accident where EDT was taken to be the ‘one-boxing’ decision theory. First, any use of probability theory in the NP with infallible predictor is suspicious, because the problem can be specified in a logically complete way with no room for empirical uncertainty. (This is why dominance reasoning is brought in for CDT. What should the probabilities be?). Second, EDT is not easy to make coherent given an agent who knows they follow EDT. (The action that EDT disfavors will have probability zero and so the agent cannot condition on it in traditional probability theory). Third, EDT just barely one-boxes. It doesn’t one-box on Double Transparent Newcomb, nor on Counterfactual Mugging. It’s also obscure what it does on PD. (Again, I can play the PD against a selfish clone of myself, with both agents having each other’s source code. There is no empirical uncertainty here, and so applying probability theory immediate raises deep foundational problems).
If TDT/UDT had come first (including the logical models and deep connections to Godel’s theorem), the philosophy discussion of NP would have been very different. EDT (which brings into the NP very dubious empirical probability distributions) would not have been considered at all for NP. I don’t see that CDT would have held much interest if its alternative was not as feeble as EDT.
It is important to understand why economists have done so much work with Nash Equilibria (e.g. on the PD) rather than invent UDT. This is explained by the fact that the assumption of logical correlation and perfect empirical knowledge between agents in the PD is not the practical reality. This doesn’t mean that UDT is not relevant to practical situations, but only that these situations involve many additional elements that may be complex to deal with in UDT. Causal based theories would have been interesting independently, for the reasons noted above concerning robust correlations.
EDIT: I realize the comment by Paul Christiano sometimes describes UDT as a variant of EDT. When I used the term “EDT” I mean the theory discussed in the philosophy literature which involves choosing the action that maximizes P(outcomes / action). This is a theory which essentially makes use of vanilla conditional probability. In what I say, I assume that UDT/TDT, despite some similarity to EDT in spirit, are not limited to regular conditioning and do not fail on smoking lesion.
I wonder if David Lewis (perhaps the most notorious philosophical two-boxer) was skeptical that any human had a sufficiently strong self-model. I think there are very who few have better self-models than he did, so it’s quite interesting if he did think this. His discussion of the “tickle defence” in his paper “Causal Decision Theory” may point that way.