The premise is that a rational agent would start out convinced that this story about the alien that knows in advance what they’ll decide appears to be false.
The Kolomogorov complexity of the story about the alien is very large because we have to hypothesize some mechanism by which it can extrapolate the contents of minds. Even if I saw the alien land a million times and watched the box-picking connect with the box contents as they’re supposed to, it is simpler to assume that the boxes are some stage magic trick, or even that they are an exception to the usual laws of physics.
Once we’ve done enough experiments that we’re forced into the hypothesis that the boxes are an exception to the usual laws of physics, it’s pretty clear what to do. The obvious revised laws of physics based on the new observations make it clear that one should choose just one box.
So a rational agent would do the right thing, but only because there’s no way to get it to believe the backstory.
The premise is that a rational agent would start out convinced that this story about the alien that knows in advance what they’ll decide appears to be false.
The Kolomogorov complexity of the story about the alien is very large because we have to hypothesize some mechanism by which it can extrapolate the contents of minds. Even if I saw the alien land a million times and watched the box-picking connect with the box contents as they’re supposed to, it is simpler to assume that the boxes are some stage magic trick, or even that they are an exception to the usual laws of physics.
Once we’ve done enough experiments that we’re forced into the hypothesis that the boxes are an exception to the usual laws of physics, it’s pretty clear what to do. The obvious revised laws of physics based on the new observations make it clear that one should choose just one box.
So a rational agent would do the right thing, but only because there’s no way to get it to believe the backstory.