You’re right, I was overstating there. I don’t think it’s probable that everything cancels out, but a more realistic statement might be something like “if UDT starts with a broad prior which wasn’t designed to address this concern, there will probably be many situations where its actions are more influenced by alternative possibilities (delusional, from our perspective) than by what it knows about the branch that it is in”.
You’re right, I was overstating there. I don’t think it’s probable that everything cancels out, but a more realistic statement might be something like “if UDT starts with a broad prior which wasn’t designed to address this concern, there will probably be many situations where its actions are more influenced by alternative possibilities (delusional, from our perspective) than by what it knows about the branch that it is in”.