Assessing by how well the decision-maker does in possible worlds that she isn’t in fact in doesn’t seem a compelling criterion (and EDT and CDT could both do well by that criterion, too, depending on which possible worlds one is allowed to pick).
You make the claim that EDT and CDT can claim optimality in exactly the same way that FDT can, here, but I think the arguments are importantly not symmetric. CDT and EDT are optimal according to their own optimality notions, but given the choice to implement different decision procedures on later problems, both the CDT and EDT optimality notions would endorse selecting FDT over themselves in many of the problems mentioned in the paper, whereas FDT will endorse itself.
Most of this section seems to me to be an argument to make careful level distinctions, in an attempt to avoid the level-crossing argument which is FDT’s main appeal. Certainly, FDTers such as myself will often use language which confuses the various levels, since we take a position which says they should be confusable—the best decision procedures should follow the best policies, which should take the best actions. But making careful level distinctions does not block the level-crossing argument, it only clarifies it. FDT may not be the only “consistent fixed-point of normativity” (to the extent that it even is that), but CDT and EDT are clearly not that.
Fourth, arguing that FDT does best in a class of ‘fair’ problems, without being able to define what that class is or why it’s interesting, is a pretty weak argument.
I basically agree that the FDT paper dropped the ball here, in that it could have given a toy setting in which ‘fair’ is rigorously defined (in a pretty standard game-theoretic setting) and FDT has the claimed optimality notion. I hope my longer writeup can make such a setting clear.
Briefly: my interpretation of the “FDT does better” claim in the FDT paper is that FDT is supposed to take UDT-optimal actions. To the extent that it doesn’t take UDT-optimal actions, I mostly don’t endorse the claim that it does better (though I plan to note in a follow-up post an alternate view in which the FDT notion of optimality may be better).
The toy setting I have in mind that makes “UDT-optimal” completely well-defined is actually fairly general. The idea is that if we can represent a decision problem as a (single-player) extensive-form game, UDT is just the idea of throwing out the requirement of subgame-optimality. In other words, we don’t even need a notion of “fairness” in the setting of extensive-form games—the setting isn’t rich enough to represent any “unfair” problems. Yet it is a pretty rich setting.
The FDT paper may have left out this model out of a desire for greater generality, which I do think is an important goal—from my perspective, it makes sense not to reduce things to the toy model in which everything works out nicely.
Response to Section VII:
You make the claim that EDT and CDT can claim optimality in exactly the same way that FDT can, here, but I think the arguments are importantly not symmetric. CDT and EDT are optimal according to their own optimality notions, but given the choice to implement different decision procedures on later problems, both the CDT and EDT optimality notions would endorse selecting FDT over themselves in many of the problems mentioned in the paper, whereas FDT will endorse itself.
Most of this section seems to me to be an argument to make careful level distinctions, in an attempt to avoid the level-crossing argument which is FDT’s main appeal. Certainly, FDTers such as myself will often use language which confuses the various levels, since we take a position which says they should be confusable—the best decision procedures should follow the best policies, which should take the best actions. But making careful level distinctions does not block the level-crossing argument, it only clarifies it. FDT may not be the only “consistent fixed-point of normativity” (to the extent that it even is that), but CDT and EDT are clearly not that.
I basically agree that the FDT paper dropped the ball here, in that it could have given a toy setting in which ‘fair’ is rigorously defined (in a pretty standard game-theoretic setting) and FDT has the claimed optimality notion. I hope my longer writeup can make such a setting clear.
Briefly: my interpretation of the “FDT does better” claim in the FDT paper is that FDT is supposed to take UDT-optimal actions. To the extent that it doesn’t take UDT-optimal actions, I mostly don’t endorse the claim that it does better (though I plan to note in a follow-up post an alternate view in which the FDT notion of optimality may be better).
The toy setting I have in mind that makes “UDT-optimal” completely well-defined is actually fairly general. The idea is that if we can represent a decision problem as a (single-player) extensive-form game, UDT is just the idea of throwing out the requirement of subgame-optimality. In other words, we don’t even need a notion of “fairness” in the setting of extensive-form games—the setting isn’t rich enough to represent any “unfair” problems. Yet it is a pretty rich setting.
This observation was already made here: https://www.lesswrong.com/posts/W4sDWwGZ4puRBXMEZ/single-player-extensive-form-games-as-a-model-of-udt. Note that there are some concerns in the comments. I think the concerns make sense, and I’m not quite sure how I want to address them, but I also don’t think they’re damning to the toy model.
The FDT paper may have left out this model out of a desire for greater generality, which I do think is an important goal—from my perspective, it makes sense not to reduce things to the toy model in which everything works out nicely.