The original version of this post claimed that an MDP-independent constant C helped lower-bound the probability assigned to power-seeking reward functions under simplicity priors. This constant is not actually MDP-independent (at least, the arguments given don’t show that): the proof sketch assumes that the MDP is given as input to the permutation-finding algorithm (which is the same, for every MDP you want to apply it to!). But the input’s description length must also be part of the Kolmogorov complexity (else you could just compute any string for free by saying “the identity program outputs the string, given the string as input”).
The upshot is that the given lower bound is weaker for more complex environments. There are other possible recourses, like “at least half of the permutations of any NPS element will be PS element, and they surely can’t all be high-complexity permutations!” — but I leave that open for now.
The original version of this post claimed that an MDP-independent constant C helped lower-bound the probability assigned to power-seeking reward functions under simplicity priors. This constant is not actually MDP-independent (at least, the arguments given don’t show that): the proof sketch assumes that the MDP is given as input to the permutation-finding algorithm (which is the same, for every MDP you want to apply it to!). But the input’s description length must also be part of the Kolmogorov complexity (else you could just compute any string for free by saying “the identity program outputs the string, given the string as input”).
The upshot is that the given lower bound is weaker for more complex environments. There are other possible recourses, like “at least half of the permutations of any NPS element will be PS element, and they surely can’t all be high-complexity permutations!” — but I leave that open for now.
Oops, and fixed.