Someone in the basement universe is reasoning about the output of a randomized Turing machine that I’m running on.
I care about what they believe about that Turing machine. Namely, I want them to believe that most of the time when the sequence x appears, it is followed by a 1.
Their beliefs depend in a linear way on my probabilities of action.
(At least if e.g. I committed to that policy at an early enough time for them to reason about it, or if my policy is sufficiently predictable to be correlated with their predictions, or if they are able to actually simulate me in a universe with reflective oracles… If I’m not able to influence their beliefs about me, then of course I can’t influence their beliefs about anything and the whole manipulative project doesn’t get off the ground.)
But my utility is a non-linear function of their beliefs, since P(1|x) is a non-linear function of their beliefs.
So my utility is a non-linear function of my policy.
If I flip a coin to randomize between two policies, I don’t see how that mixed policy could produce more value for me than the base policies.
(ETA: the logical implications about the fact of my randomization don’t have any weird anti-adversarial effects here).
Someone in the basement universe is reasoning about the output of a randomized Turing machine that I’m running on.
I care about what they believe about that Turing machine. Namely, I want them to believe that most of the time when the sequence x appears, it is followed by a 1.
Their beliefs depend in a linear way on my probabilities of action.
(At least if e.g. I committed to that policy at an early enough time for them to reason about it, or if my policy is sufficiently predictable to be correlated with their predictions, or if they are able to actually simulate me in a universe with reflective oracles… If I’m not able to influence their beliefs about me, then of course I can’t influence their beliefs about anything and the whole manipulative project doesn’t get off the ground.)
But my utility is a non-linear function of their beliefs, since P(1|x) is a non-linear function of their beliefs.
So my utility is a non-linear function of my policy.