One would have to ask Eliezer and Nat what they really meant, since it is easy to end up in a self-contradictory setup or to ask a question about an impossible world, like to asking what happens if in the Newcomb’s setup the agent decided to switch to two-boxing after the perfect predictor had already put $1,000,000 in.
My wild guess is that the FDT Fiona from the paper uses a certain decision theory DT1 that does not cope well with the world with adversarial predictors. She uses some kind of causal decision graph logic that would lead her astray instead of being in the winning world. I also assume that Fiona makes her “decisions” while being fully informed about the predictor’s intentions to punish her and just CDT-like throws her hands in the air and cries “unfair!”
One would have to ask Eliezer and Nat what they really meant, since it is easy to end up in a self-contradictory setup or to ask a question about an impossible world, like to asking what happens if in the Newcomb’s setup the agent decided to switch to two-boxing after the perfect predictor had already put $1,000,000 in.
My wild guess is that the FDT Fiona from the paper uses a certain decision theory DT1 that does not cope well with the world with adversarial predictors. She uses some kind of causal decision graph logic that would lead her astray instead of being in the winning world. I also assume that Fiona makes her “decisions” while being fully informed about the predictor’s intentions to punish her and just CDT-like throws her hands in the air and cries “unfair!”