I prefer this interpretation: P(a=X) means how sure the agent is it will X. If it flips a coin do decide whether X or Y, P(a=X)=P(a=Y)~=0.5. If it’s chosen to “just X”, P(a=X) ~= 1. Omega for his part knows the agent’s surety and uses a randomizing device to match his actions with it.
ETA: if interpreted naively, this leads to Omega rewarding agents with deluded beliefs about what they’re going to do. Maybe Omega shouldn’t look at the agent’s surety but the surety of “a perfectly rational agent” in the same situation. I don’t have a real solution to this right now.
I prefer this interpretation: P(a=X) means how sure the agent is it will X. If it flips a coin do decide whether X or Y, P(a=X)=P(a=Y)~=0.5. If it’s chosen to “just X”, P(a=X) ~= 1. Omega for his part knows the agent’s surety and uses a randomizing device to match his actions with it.
ETA: if interpreted naively, this leads to Omega rewarding agents with deluded beliefs about what they’re going to do. Maybe Omega shouldn’t look at the agent’s surety but the surety of “a perfectly rational agent” in the same situation. I don’t have a real solution to this right now.