If Y is sufficiently powerful, it may be able to deduce the laws of physics and the initial conditions of the universe, and then obtain X’s source code by simulating the universe up to when X is created.
Unless there have been some specific assumptions made about the universe that will not work. Simulating the entire universe does not tell Y which part of the universe it inhabits. It will give Y a set of possible parts of the universe which match Y’s observations. While the simulation strategy will allow the best possible prediction about what X’s source code is given what Y already knows it does not give evidence to Y that it didn’t already have.
You’re right, the model assumes that we live in a universe such that superintelligent AIs would “naturally” have enough evidence to infer the source code of other AIs. (That seems quite plausible, although by no means certain, to me.) Also, since this is a thread about the relative merits of CDT, I should point out that there are some games in which CDT seems to win relative to TDT or UDT, which is a puzzle that is still open.
Also, since this is a thread about the relative merits of CDT, I should point out that there are some games in which CDT seems to win relative to TDT or UDT, which is a puzzle that is still open.
It’s an interesting problem, but my impression when reading was somewhat similar to that of Eliezer in the replies. At the core it is the question of “How do you deal with constructs made by other agents?” I don’t think TDT has any particular weakness there.
Unless there have been some specific assumptions made about the universe that will not work. Simulating the entire universe does not tell Y which part of the universe it inhabits. It will give Y a set of possible parts of the universe which match Y’s observations. While the simulation strategy will allow the best possible prediction about what X’s source code is given what Y already knows it does not give evidence to Y that it didn’t already have.
You’re right, the model assumes that we live in a universe such that superintelligent AIs would “naturally” have enough evidence to infer the source code of other AIs. (That seems quite plausible, although by no means certain, to me.) Also, since this is a thread about the relative merits of CDT, I should point out that there are some games in which CDT seems to win relative to TDT or UDT, which is a puzzle that is still open.
It’s an interesting problem, but my impression when reading was somewhat similar to that of Eliezer in the replies. At the core it is the question of “How do you deal with constructs made by other agents?” I don’t think TDT has any particular weakness there.