You raise an interesting question here—what would CDT do if a CDT agent were in the simulation?
It looks to me that CDT just doesn’t have the conceptual machinery to handle this problem properly, so I don’t really know.
One thing that could happen is that the simulated CDT agent tries to simulate itself and gets stuck in an infinite loop. I didn’t specify exactly what would happen in that case, but if Omega can prove that the simulated agent is caught in a loop, then it knows the sim will choose each box with probability zero, and so (since these are all equal), it will fill box 1. But now can a real-life CDT agent also work this out, and beat the game by selecting box 1. But if so, why won’t the sim do that, and so on? Aargh !!!
Another thought I had is that CDT could try tossing a logical coin, like computing the googleth digit of pi, and if it is even choose box 1, whereas if it is odd, choose box 2. If it runs out of time before computing (which the real-life agent will do), then it just picks box 1 or 2 with equal probability. The simulated CDT agent will however get to the end of the computation (Omega has arbitrary computational resources) and definitely pick 1 or 2 with certainty, so the money is definitely in one of those two boxes, which looks like the probability of the actual agent winning is raised to 50%. TDT might do the same.
However this looks like cheating to me, for both CDT and TDT.
EDIT: On reflection, it seems clear that CDT would never do anything “creatively sneaky” like tossing a logical coin; but it is the sort of approach that TDT (or some variant thereof) might come up with. Though I still think it’s cheating.
I don’t think your “detect infinite resources and cheat” strategy is really worth thinking about. Instead of strategies like CDT and TDT whose applicability to limited compute resources is unclear, suppose you have an anytime strategy X, which you can halt at any time and get a decision. Then there’s really a family of algorithms X-t, where t is the time you’re going to give it to run. In this case, if you are X-t, we can consider the situation where Omega fields X-t against you.
The version of CDT that I described explicitly should arrive at the uniformly random solution. You don’t have to be able to simulate a program all the way through, just able to prove things about its output.
EDIT: Wait, this is wrong. It won’t be able to consistently derive an answer, because of the way it acts given such an answer, and so it will go with whatever its default Nash equilibrium is.
Re: your EDIT. Yes, I’ve had that sort of reaction a couple of times today!
I’m shifting around between “CDT should pick at random, no CDT should pick Box 1, no CDT should use a logical coin, no CDT should pick it’s favourite number in the set {1, 2} with probability 1, and hope that the version in the sim has a different favourite number, no, CDT will just go into a loop or collapse in a heap.”
I’m also quite clueless how a TDT is supposed to decide if it’s told there’s a CDT in the sim… This looks like a pretty evil decision problem in its own right.
Well, the thing is that CDT doesn’t completely specify a decision theory. I’m confident now that the specific version of CDT that I described would fail to deduce anything and go with its default, but it’s hard to speak for CDTs in general on such a self-referential problem.
You raise an interesting question here—what would CDT do if a CDT agent were in the simulation?
It looks to me that CDT just doesn’t have the conceptual machinery to handle this problem properly, so I don’t really know. One thing that could happen is that the simulated CDT agent tries to simulate itself and gets stuck in an infinite loop. I didn’t specify exactly what would happen in that case, but if Omega can prove that the simulated agent is caught in a loop, then it knows the sim will choose each box with probability zero, and so (since these are all equal), it will fill box 1. But now can a real-life CDT agent also work this out, and beat the game by selecting box 1. But if so, why won’t the sim do that, and so on? Aargh !!!
Another thought I had is that CDT could try tossing a logical coin, like computing the googleth digit of pi, and if it is even choose box 1, whereas if it is odd, choose box 2. If it runs out of time before computing (which the real-life agent will do), then it just picks box 1 or 2 with equal probability. The simulated CDT agent will however get to the end of the computation (Omega has arbitrary computational resources) and definitely pick 1 or 2 with certainty, so the money is definitely in one of those two boxes, which looks like the probability of the actual agent winning is raised to 50%. TDT might do the same.
However this looks like cheating to me, for both CDT and TDT.
EDIT: On reflection, it seems clear that CDT would never do anything “creatively sneaky” like tossing a logical coin; but it is the sort of approach that TDT (or some variant thereof) might come up with. Though I still think it’s cheating.
I don’t think your “detect infinite resources and cheat” strategy is really worth thinking about. Instead of strategies like CDT and TDT whose applicability to limited compute resources is unclear, suppose you have an anytime strategy X, which you can halt at any time and get a decision. Then there’s really a family of algorithms X-t, where t is the time you’re going to give it to run. In this case, if you are X-t, we can consider the situation where Omega fields X-t against you.
The version of CDT that I described explicitly should arrive at the uniformly random solution. You don’t have to be able to simulate a program all the way through, just able to prove things about its output.
EDIT: Wait, this is wrong. It won’t be able to consistently derive an answer, because of the way it acts given such an answer, and so it will go with whatever its default Nash equilibrium is.
Re: your EDIT. Yes, I’ve had that sort of reaction a couple of times today!
I’m shifting around between “CDT should pick at random, no CDT should pick Box 1, no CDT should use a logical coin, no CDT should pick it’s favourite number in the set {1, 2} with probability 1, and hope that the version in the sim has a different favourite number, no, CDT will just go into a loop or collapse in a heap.”
I’m also quite clueless how a TDT is supposed to decide if it’s told there’s a CDT in the sim… This looks like a pretty evil decision problem in its own right.
Well, the thing is that CDT doesn’t completely specify a decision theory. I’m confident now that the specific version of CDT that I described would fail to deduce anything and go with its default, but it’s hard to speak for CDTs in general on such a self-referential problem.