For counterlogical mugging, it’s unclear if it should be possible to correctly discover the parity of the relevant digit of pi. I would expect that in the counterfactual where it’s even, it will eventually be discovered to be even. And in the countefactual where it’s odd, that same digit will eventually be discovered to be odd.
ASP and Transparent Newcomb might be closer to test cases for formulating updateless policies that have the character of getting better as they grow more powerful. These problems ask the agent to use a decision procedure that intentionally doesn’t take certain information into account, whether the agent as a whole has access to that information or not. But they lack future steps that would let that decision procedure benefit from eventually getting stronger than the agent that initially formulated it, so these aren’t quite the thought experiments needed here.
For counterlogical mugging, it’s unclear if it should be possible to correctly discover the parity of the relevant digit of pi. I would expect that in the counterfactual where it’s even, it will eventually be discovered to be even. And in the countefactual where it’s odd, that same digit will eventually be discovered to be odd.
ASP and Transparent Newcomb might be closer to test cases for formulating updateless policies that have the character of getting better as they grow more powerful. These problems ask the agent to use a decision procedure that intentionally doesn’t take certain information into account, whether the agent as a whole has access to that information or not. But they lack future steps that would let that decision procedure benefit from eventually getting stronger than the agent that initially formulated it, so these aren’t quite the thought experiments needed here.