This might not satisfactorily answer your confusion but: CDT is defined by the fact that it has incorrect causal graphs. If it has correct causal graphs then it’s not CDT. Why bother talking about a “decision theory” that is arbitrarily limited to incorrect causal graphs? Because that’s the decision theory that academic decision theorists like to talk about and treat as default. Why did academic decision theorists never realize that their causal graphs were wrong? No one has a very good model of that, but check out Wei Dai’s related speculation here. Note that if you define causality in a technical Markovian way and use Bayes nets then there is no difference between CDT and TDT.
I used to get annoyed because CDT with a good enough world model should clearly one-box yet people stipulated that it wouldn’t; only later did I realize that it’s mostly a rhetorical thing and no one thinks that if you actually implemented an AGI with “CDT” that it’d be as dumb as academia/LessWrong’s version of CDT.
If I’m wrong about any of the above then someone please correct me as this is relevant to FAI strategy.
This might not satisfactorily answer your confusion but: CDT is defined by the fact that it has incorrect causal graphs. If it has correct causal graphs then it’s not CDT. Why bother talking about a “decision theory” that is arbitrarily limited to incorrect causal graphs? Because that’s the decision theory that academic decision theorists like to talk about and treat as default. Why did academic decision theorists never realize that their causal graphs were wrong? No one has a very good model of that, but check out Wei Dai’s related speculation here. Note that if you define causality in a technical Markovian way and use Bayes nets then there is no difference between CDT and TDT.
I used to get annoyed because CDT with a good enough world model should clearly one-box yet people stipulated that it wouldn’t; only later did I realize that it’s mostly a rhetorical thing and no one thinks that if you actually implemented an AGI with “CDT” that it’d be as dumb as academia/LessWrong’s version of CDT.
If I’m wrong about any of the above then someone please correct me as this is relevant to FAI strategy.