This is similar to the ASP problem, an unusual anthropic use case. The issue with UDT is that it’s underspecified for such cases, but I think some of its concepts are still clearer than the classical probability/causality language.
UDT can be reframed in the following way. There is an abstract agent that’s not part of any real world of interest, which is just a process that runs according to its program and can’t be disrupted with an anvil dropped on its head. It covers all possibilities, so it includes more than one history. Worlds can “incarnate” parts of this process, either directly, by straightforward interpretation of its program with particular observations fed to it, or indirectly, by reasoning about it. As a result, certain events in certain worlds are controlled by the abstract process through such incarnations. (This imagery doesn’t apply to PD though, where the controlled thing is not an event in a world; this restriction puts it closer to what TDT does, whereas proof-based UDT is more general.)
The normal way of describing UDT’s algorithm (in this restricted form) is that there are three phases. In the first phase, usually screened off by the problem statement, the agent identifies the events in the worlds of interest that it controls. Then, in the second phase, it examines the consequences of the possible action strategies, and selects a strategy. In the third phase, it enacts the strategy, selecting a concrete action depending on observations.
The problem with this in anthropic problems, such as ASP and your Coin Flip Creation problem, is that strategy-selection and action-selection can affect which events are influenced by incarnations of the agent. Some of the computations that could be performed on any of the phases make it impossible to incarnate the agent in some of the situations where it would otherwise get to be incarnated, so the results of the first phase can depend on how the agent is thinking on the subsequent phases. For example, if the agent is just simulated to completion, then it loses access to the action if it takes too long to complete. This also applies to abstract reasoning about the agent, where it can diagonalize that reasoning to make it impossible.
So an agent should sometimes decide how to think, in a way that doesn’t discourage too many situations in the worlds where it’s thinking that. This creates additional problems (different agents that have to think differently, unlike the unified UDT), but that’s outside the scope of this post. For ASP, the trick is to notice how simple its thinking has to be to retain control over Predictor’s prediction, and to make the decision within that constraint.
For Coin Flip Creation, an agent that decides counter to its gene doesn’t get to inhabit the world with that gene, since there is no difference between the decision making setups in the two worlds other than the agents who are making the decision. The agent will be “eliminated” by Omega from the world whose gene is different from the agent’s decision (i.e. not allowed to reach the decision making setup, via an arrangement of the initial conditions), and instead a different agent will be put in control in that world. So one-boxing makes the two-box gene world inaccessible to the agent, and conversely. Since I assume randomizing is impossible or punished in some way, the choice is really between which world the agent will inhabit, in which case the one-box world seems a bit better (the other world will be inhabited by an agent with a different decision theory, possibly a crazier one, less capable of putting money to good use). If the agent is “altruistic” and doesn’t expect much difference in how its counterpart will manage its funds, the choice doesn’t matter. On the other hand, if the agent were told its gene, then it should just go with it (act according to the gene), since that will give it access to both worlds (in this case, it doesn’t matter at all what’s in the boxes).
Thanks for your comment! I find your line of reasoning in the ASP problem and the Coin Flip Creation plausible. So your point is that, in both cases, by choosing a decision algorithm, one also gets to choose where this algorithm is being instantiated? I would say that in the CFC, choosing the right action is sufficient, while in the ASP you also have to choose the whole UDP program so as to be instantiated in a beneficial way (similar to the distinction of how TDT iterates over acts and UDT iterates over policies).
Would you agree that the Coin Flip Creation is similar to e.g. the Smoking Lesion? I could also imagine that by not smoking, UDT would become more likely to be instantiated in a world where the UDT agent doesn’t have the gene (or that the gene would eliminate (some of) the UDT agents from the worlds where they have cancer). Otherwise there couldn’t be a study showing a correlation between UDT agents’ genes and their smoking habits. If the participants of the study used a different decision theory or, unlike us, didn’t have knowledge of the results of the study, UDT would probably smoke. But in this case I would argue that EDT would do so as well, since conditioning on all of this information puts it out of the reference class of the people in the study.
One could probably generalize this kind of “likelihood of being instantiated” reasoning. My guess would be that an UDT version that takes it into account might behave according to conditional probabilities like EDT. Take e.g. the example from this post by Nate Soares. If there isn’t a principled difference to the Coin Flip Case that I’ve overlooked, then UDT might reason that if it takes “green”, it will become very likely that it will be instantiated only in a world where gamma rays hit the UDT agent (since apparently, UDT agents that choose green are “eliminated” from worlds without gamma rays – or at least that’s what I have to assume if I don’t know any additional facts). Therefore our specified version of UDT takes the red box. The main argument I’m trying to make is that if you solve the problem like this, then UDT would (at least here, and possibly in all cases) become equivalent to updateless EDT. Which as far as I know would be a relief, since (u)EDT seems easier to formalize?
So your point is that, in both cases, by choosing a decision algorithm, one also gets to choose where this algorithm is being instantiated?
To clarify, it’s the algorithm itself that chooses how it behaves. So I’m not talking about how algorithm’s instantiation depends on the way programmer chooses to write it, instead I’m talking about how algorithm’s instantiation depends on the choices that the algorithm itself makes, where we are talking about a particular algorithm that’s already written. Less mysteriously, the idea of algorithm’s decisions influencing things describes a step in the algorithm, it’s how the algorithm operates, by figuring out something we could call “how algorithm’s decisions influence outcomes”. The algorithm then takes that thing and does further computations that depend on it.
This is similar to the ASP problem, an unusual anthropic use case. The issue with UDT is that it’s underspecified for such cases, but I think some of its concepts are still clearer than the classical probability/causality language.
UDT can be reframed in the following way. There is an abstract agent that’s not part of any real world of interest, which is just a process that runs according to its program and can’t be disrupted with an anvil dropped on its head. It covers all possibilities, so it includes more than one history. Worlds can “incarnate” parts of this process, either directly, by straightforward interpretation of its program with particular observations fed to it, or indirectly, by reasoning about it. As a result, certain events in certain worlds are controlled by the abstract process through such incarnations. (This imagery doesn’t apply to PD though, where the controlled thing is not an event in a world; this restriction puts it closer to what TDT does, whereas proof-based UDT is more general.)
The normal way of describing UDT’s algorithm (in this restricted form) is that there are three phases. In the first phase, usually screened off by the problem statement, the agent identifies the events in the worlds of interest that it controls. Then, in the second phase, it examines the consequences of the possible action strategies, and selects a strategy. In the third phase, it enacts the strategy, selecting a concrete action depending on observations.
The problem with this in anthropic problems, such as ASP and your Coin Flip Creation problem, is that strategy-selection and action-selection can affect which events are influenced by incarnations of the agent. Some of the computations that could be performed on any of the phases make it impossible to incarnate the agent in some of the situations where it would otherwise get to be incarnated, so the results of the first phase can depend on how the agent is thinking on the subsequent phases. For example, if the agent is just simulated to completion, then it loses access to the action if it takes too long to complete. This also applies to abstract reasoning about the agent, where it can diagonalize that reasoning to make it impossible.
So an agent should sometimes decide how to think, in a way that doesn’t discourage too many situations in the worlds where it’s thinking that. This creates additional problems (different agents that have to think differently, unlike the unified UDT), but that’s outside the scope of this post. For ASP, the trick is to notice how simple its thinking has to be to retain control over Predictor’s prediction, and to make the decision within that constraint.
For Coin Flip Creation, an agent that decides counter to its gene doesn’t get to inhabit the world with that gene, since there is no difference between the decision making setups in the two worlds other than the agents who are making the decision. The agent will be “eliminated” by Omega from the world whose gene is different from the agent’s decision (i.e. not allowed to reach the decision making setup, via an arrangement of the initial conditions), and instead a different agent will be put in control in that world. So one-boxing makes the two-box gene world inaccessible to the agent, and conversely. Since I assume randomizing is impossible or punished in some way, the choice is really between which world the agent will inhabit, in which case the one-box world seems a bit better (the other world will be inhabited by an agent with a different decision theory, possibly a crazier one, less capable of putting money to good use). If the agent is “altruistic” and doesn’t expect much difference in how its counterpart will manage its funds, the choice doesn’t matter. On the other hand, if the agent were told its gene, then it should just go with it (act according to the gene), since that will give it access to both worlds (in this case, it doesn’t matter at all what’s in the boxes).
Thanks for your comment! I find your line of reasoning in the ASP problem and the Coin Flip Creation plausible. So your point is that, in both cases, by choosing a decision algorithm, one also gets to choose where this algorithm is being instantiated? I would say that in the CFC, choosing the right action is sufficient, while in the ASP you also have to choose the whole UDP program so as to be instantiated in a beneficial way (similar to the distinction of how TDT iterates over acts and UDT iterates over policies).
Would you agree that the Coin Flip Creation is similar to e.g. the Smoking Lesion? I could also imagine that by not smoking, UDT would become more likely to be instantiated in a world where the UDT agent doesn’t have the gene (or that the gene would eliminate (some of) the UDT agents from the worlds where they have cancer). Otherwise there couldn’t be a study showing a correlation between UDT agents’ genes and their smoking habits. If the participants of the study used a different decision theory or, unlike us, didn’t have knowledge of the results of the study, UDT would probably smoke. But in this case I would argue that EDT would do so as well, since conditioning on all of this information puts it out of the reference class of the people in the study.
One could probably generalize this kind of “likelihood of being instantiated” reasoning. My guess would be that an UDT version that takes it into account might behave according to conditional probabilities like EDT. Take e.g. the example from this post by Nate Soares. If there isn’t a principled difference to the Coin Flip Case that I’ve overlooked, then UDT might reason that if it takes “green”, it will become very likely that it will be instantiated only in a world where gamma rays hit the UDT agent (since apparently, UDT agents that choose green are “eliminated” from worlds without gamma rays – or at least that’s what I have to assume if I don’t know any additional facts). Therefore our specified version of UDT takes the red box. The main argument I’m trying to make is that if you solve the problem like this, then UDT would (at least here, and possibly in all cases) become equivalent to updateless EDT. Which as far as I know would be a relief, since (u)EDT seems easier to formalize?
To clarify, it’s the algorithm itself that chooses how it behaves. So I’m not talking about how algorithm’s instantiation depends on the way programmer chooses to write it, instead I’m talking about how algorithm’s instantiation depends on the choices that the algorithm itself makes, where we are talking about a particular algorithm that’s already written. Less mysteriously, the idea of algorithm’s decisions influencing things describes a step in the algorithm, it’s how the algorithm operates, by figuring out something we could call “how algorithm’s decisions influence outcomes”. The algorithm then takes that thing and does further computations that depend on it.