Seems like a simple and reasonable answer to this problem is that I would take the box with the million dollars, rather than the box with the thousand dollars and the empty box. It seems the main question is, “But why?”. So here is my reasoning: Omega has shown over 99% accuracy in providing results dependent on people’s choices. Box B has 100,000% better rewards than Box A, such that if there is even 0.1% chance that taking Box A will lose those rewards, it is irrational to also take Box A. As I have seen no evidence that Omega has left, it is not even certain that my choice of actions now will have not effect on the contents of the opaque box (only a fool would be certain that just because he “saw Omega fly away” that said superintelligence is not hiding nor has left behind an observing agent). As each of these choices would cause my choice to be only Box B, it is almost certain that Omega has seen likewise and put the $1,000,000 in Box B.
I suspect the problem people seem to have with this is because they think they are outside of the game. But the game description itself says that (a very accurate model of) you is in the game, and that therefore your (modeled) choices, including second doubts and your desire not to leave that last $1,000, will (if modeled correctly) affect the contents of Box B. No, Omega is not rewarding irrationality. Omega is giving a large reward to those who trust in Omega’s judgement, and a smaller reward to those who arrogantly think they can cheat the game he set up.
Seems like a simple and reasonable answer to this problem is that I would take the box with the million dollars, rather than the box with the thousand dollars and the empty box. It seems the main question is, “But why?”. So here is my reasoning: Omega has shown over 99% accuracy in providing results dependent on people’s choices. Box B has 100,000% better rewards than Box A, such that if there is even 0.1% chance that taking Box A will lose those rewards, it is irrational to also take Box A. As I have seen no evidence that Omega has left, it is not even certain that my choice of actions now will have not effect on the contents of the opaque box (only a fool would be certain that just because he “saw Omega fly away” that said superintelligence is not hiding nor has left behind an observing agent). As each of these choices would cause my choice to be only Box B, it is almost certain that Omega has seen likewise and put the $1,000,000 in Box B.
I suspect the problem people seem to have with this is because they think they are outside of the game. But the game description itself says that (a very accurate model of) you is in the game, and that therefore your (modeled) choices, including second doubts and your desire not to leave that last $1,000, will (if modeled correctly) affect the contents of Box B. No, Omega is not rewarding irrationality. Omega is giving a large reward to those who trust in Omega’s judgement, and a smaller reward to those who arrogantly think they can cheat the game he set up.