This proves too much. For any thought experiment, if you are allowed to introduce a generalized Nomega, you can force any conclusion for an agent that cares about counterfactuals or has a chance to make a global precommitment. What if Nomega pays $50000 when you do precommit to pay $100 to Omega? Or when you Defect in Prisoner’s Dilemma? Or when you Cooperate in Prisoner’s Dilemma? This shows that if Nomega is allowed to be introduced, none of the usual decision theory thought experiments can be usefully considered (with a theory like UDT). Thus, it’s a reasonable assumption that Nomega shouldn’t be allowed to be introduced.
(Btw, the expected values you list in the table are off. Don’t know what’s up with that.)
Upon further reflection, I think the crux of my argument is that by precommiting you are essentially pascals wagering yourself—you are making a decision looking to maximize yoir reward should a certain type of God (Omega) exist. Unless (before you get mugged) you have some reason to believe that this type of God is more likley to exist then the opposite type (Nomega), then precommiting is getting wagered (as far as I can tell). You cant wait until you find out that Omega exists to preccomit because by then you have aready learned that the coin is tails—you have to do so blind.
I dont think this proves to much becuase in other problems (Prisoners Dilema, Newcombs Paradox, etc.) considering what to do if a random God shows is wagering, so you just ignore it. Here, though, precommiting is wagering, so (it seems to me) that youshould just ignore it as well and not precommit.
Good point on EV numbers—they are now updated although the actual numbers are not super important to the crux of the argument.
I think I see what you mean. The situation where you’d make a precommitment, which is described by the same state of knowledge that UDT makes its decision under, occurs before the setting of the thought experiment is made clear. Thus it’s not yet clear what kinds of Nomegas can show up with their particular incentives, and the precommitment can’t rely on their absense. With some sort of risk-averse status-quo-anchored attitude it seems like “not precomitting” is therefore generally preferable.
But optimization of expected utility doesn’t work like that. You have the estimates for possible decisions, and pick the option that’s estimated to be the best available. Whether it’s the status quo (“not precommitting”) or not has no bearing on the decision unless it’s expressed as a change in the esimate of expected utility that makes it lower or greater than expected utility of the alternative decisions. Thus when a thought experiment talks about precommitments or any UDT decisions, bringing in arbitrary Nomegas is a problem because it makes the expected utility of precommitments similarly arbitrary, and it’s these expected utilities that determine decisions. (Whether to make some precommitment or not is itself a decision.) The obvious way of making it possible to perform the calculation of expected utilities of precommitments is to make the assumption of absense of Nomegas, or more generally to construct the settings of precommitments based only on what’s already in the thought experiment.
(Mistakes in expected values in the post are a tiny bit relevant (one of the values is still wrong after the correction), as they vaguely signal lack of reliable knowledge of what expected utility is, although the issue here is mostly informal and correct calculation won’t by itself make things clear. General experience with mathematical proofs might be closer to being helpful, as the issue is that the actual algorithms being discussed screen off a lot of informal considerations such as whether not making precommitments is the status quo.)
Perhaps I am misunderstanding the setup of the counterfactual mugging—do we live in a world in which Omega is a known being (and just hasn’t yet interacted with us), or do we live in a world in which we have roughly equal credence of the existence of Omega vs Nomega (vs any other arbitrary God-like figure). If it’s the former, then sure UDT says precommit and pay.
But if its the latter, I still don’t see why UDT tells us to pay—not because not precommitting is some sort of default (which is I agree UDT says isn’t relevant) but because when making decisions based on the possible existence of some sort of God while ignoring the possible existence of other God’s isn’t fair or an effective way to maximize you expected utility. Perhaps some sort of Occam’s Razor / Solomon Induction argument could be made that Omega is simpler and thus more likely to exist, but this seems fairly difficult to rigorize.
In the first approximation, the point is not that counterfactual mugging (or any other thought experiment) is actually defined in a certain way, but how it should be redefined in order to make it possible to navigate the issue. Unless Nomegas are outlawed, it’s not possible to do any calculations, therefore they are outlawed. Not because they were already explicitly outlawed or were colloquially understood to be outlawed.
But when we look at this more carefully, the assumption is not actually needed. If nonspecified Nomegas are allowed, the distribution of their possible incentives is all over the place, so they almost certainly cancel out in the expected utility of alternative precommitments. The real problem is not with introduction of Nomegas, but with managing to include the possibilities involving Omega in the calculations (as opposed to discarding them as particular Nomegas), taking into account the setting that’s not yet described at the point where precommitment should be made.
In counterfactual mugging, there is no physical time when the agent is in the state of knowledge where the relevant precommitment can be made (that’s the whole point). Instead, we can construct a hypothetical state of knowledge that has updated on the description of the thought experiment, but hasn’t updated on the fact of how the coin toss turned out. The agent never holds this state of knowledge as a description of all that’s actually known. Why retract knowledge of the coin toss, instead of retracting knowledge of the thought experiment? No reason, UDT strives to retract all knowledge and make a completely general precommitment to all eventualities. But in this setting, retracting knowledge of the coin toss while retaining knowledge of Omega creates a tractable decision problem, thus UDT that notices the possibility will make a precommitment. Similarly, it should precommit to not paying Omega in a situation where a Nomega punishing for paying up $100 to Omega (as described in this post) is known to operate. But only when it’s known to be there, not when it’s not known to be there.
Hmm perhaps I am still a little confused as to how UDT works. My understanding is that you don’t make your decisions based on the information you have observed, but instead, when you “boot up” your UDT, you consider all of the possible world states you may find yourself in and their various mesures, and then for each decision, “precommit” to making the one that maximizes your expected utility across all of the possible world states that this decision affects.
If this understanding is correct, then unless we have some sort of prior telling us, when we “boot up” UDT and thus before we interact with Omega, that Omega is more likley to exist than Nomega, then I don’t see how UDT could tell us to pay up.
I think it is somewhat likley that I am missing something here but I dont know what.
This proves too much. For any thought experiment, if you are allowed to introduce a generalized Nomega, you can force any conclusion for an agent that cares about counterfactuals or has a chance to make a global precommitment. What if Nomega pays $50000 when you do precommit to pay $100 to Omega? Or when you Defect in Prisoner’s Dilemma? Or when you Cooperate in Prisoner’s Dilemma? This shows that if Nomega is allowed to be introduced, none of the usual decision theory thought experiments can be usefully considered (with a theory like UDT). Thus, it’s a reasonable assumption that Nomega shouldn’t be allowed to be introduced.
(Btw, the expected values you list in the table are off. Don’t know what’s up with that.)
Hi Vladimir, thanks for your response.
Upon further reflection, I think the crux of my argument is that by precommiting you are essentially pascals wagering yourself—you are making a decision looking to maximize yoir reward should a certain type of God (Omega) exist. Unless (before you get mugged) you have some reason to believe that this type of God is more likley to exist then the opposite type (Nomega), then precommiting is getting wagered (as far as I can tell). You cant wait until you find out that Omega exists to preccomit because by then you have aready learned that the coin is tails—you have to do so blind.
I dont think this proves to much becuase in other problems (Prisoners Dilema, Newcombs Paradox, etc.) considering what to do if a random God shows is wagering, so you just ignore it. Here, though, precommiting is wagering, so (it seems to me) that youshould just ignore it as well and not precommit.
Good point on EV numbers—they are now updated although the actual numbers are not super important to the crux of the argument.
I think I see what you mean. The situation where you’d make a precommitment, which is described by the same state of knowledge that UDT makes its decision under, occurs before the setting of the thought experiment is made clear. Thus it’s not yet clear what kinds of Nomegas can show up with their particular incentives, and the precommitment can’t rely on their absense. With some sort of risk-averse status-quo-anchored attitude it seems like “not precomitting” is therefore generally preferable.
But optimization of expected utility doesn’t work like that. You have the estimates for possible decisions, and pick the option that’s estimated to be the best available. Whether it’s the status quo (“not precommitting”) or not has no bearing on the decision unless it’s expressed as a change in the esimate of expected utility that makes it lower or greater than expected utility of the alternative decisions. Thus when a thought experiment talks about precommitments or any UDT decisions, bringing in arbitrary Nomegas is a problem because it makes the expected utility of precommitments similarly arbitrary, and it’s these expected utilities that determine decisions. (Whether to make some precommitment or not is itself a decision.) The obvious way of making it possible to perform the calculation of expected utilities of precommitments is to make the assumption of absense of Nomegas, or more generally to construct the settings of precommitments based only on what’s already in the thought experiment.
(Mistakes in expected values in the post are a tiny bit relevant (one of the values is still wrong after the correction), as they vaguely signal lack of reliable knowledge of what expected utility is, although the issue here is mostly informal and correct calculation won’t by itself make things clear. General experience with mathematical proofs might be closer to being helpful, as the issue is that the actual algorithms being discussed screen off a lot of informal considerations such as whether not making precommitments is the status quo.)
Whoops—EV re-updated.
Perhaps I am misunderstanding the setup of the counterfactual mugging—do we live in a world in which Omega is a known being (and just hasn’t yet interacted with us), or do we live in a world in which we have roughly equal credence of the existence of Omega vs Nomega (vs any other arbitrary God-like figure). If it’s the former, then sure UDT says precommit and pay.
But if its the latter, I still don’t see why UDT tells us to pay—not because not precommitting is some sort of default (which is I agree UDT says isn’t relevant) but because when making decisions based on the possible existence of some sort of God while ignoring the possible existence of other God’s isn’t fair or an effective way to maximize you expected utility. Perhaps some sort of Occam’s Razor / Solomon Induction argument could be made that Omega is simpler and thus more likely to exist, but this seems fairly difficult to rigorize.
In the first approximation, the point is not that counterfactual mugging (or any other thought experiment) is actually defined in a certain way, but how it should be redefined in order to make it possible to navigate the issue. Unless Nomegas are outlawed, it’s not possible to do any calculations, therefore they are outlawed. Not because they were already explicitly outlawed or were colloquially understood to be outlawed.
But when we look at this more carefully, the assumption is not actually needed. If nonspecified Nomegas are allowed, the distribution of their possible incentives is all over the place, so they almost certainly cancel out in the expected utility of alternative precommitments. The real problem is not with introduction of Nomegas, but with managing to include the possibilities involving Omega in the calculations (as opposed to discarding them as particular Nomegas), taking into account the setting that’s not yet described at the point where precommitment should be made.
In counterfactual mugging, there is no physical time when the agent is in the state of knowledge where the relevant precommitment can be made (that’s the whole point). Instead, we can construct a hypothetical state of knowledge that has updated on the description of the thought experiment, but hasn’t updated on the fact of how the coin toss turned out. The agent never holds this state of knowledge as a description of all that’s actually known. Why retract knowledge of the coin toss, instead of retracting knowledge of the thought experiment? No reason, UDT strives to retract all knowledge and make a completely general precommitment to all eventualities. But in this setting, retracting knowledge of the coin toss while retaining knowledge of Omega creates a tractable decision problem, thus UDT that notices the possibility will make a precommitment. Similarly, it should precommit to not paying Omega in a situation where a Nomega punishing for paying up $100 to Omega (as described in this post) is known to operate. But only when it’s known to be there, not when it’s not known to be there.
Hmm perhaps I am still a little confused as to how UDT works. My understanding is that you don’t make your decisions based on the information you have observed, but instead, when you “boot up” your UDT, you consider all of the possible world states you may find yourself in and their various mesures, and then for each decision, “precommit” to making the one that maximizes your expected utility across all of the possible world states that this decision affects.
If this understanding is correct, then unless we have some sort of prior telling us, when we “boot up” UDT and thus before we interact with Omega, that Omega is more likley to exist than Nomega, then I don’t see how UDT could tell us to pay up.
I think it is somewhat likley that I am missing something here but I dont know what.