Yes, that there can just as easily be a superintelligence that rewards people predicted to act one way as one that rewards people predicted to act the other. Which precommitment is most rational depends depends on the which type you expect to encounter.
I don’t expect to encounter either, and on the other hand I can’t rule out fallible human analogues of either. So for now I’m not precommitting either way.
You don’t precommit to “give away the $100, to anyone who asks”. You precommit to give away the $100 in exactly the situation I described. Or, generalizing such precommitments, you just compute your decisions on the spot, in a reflectively consistent fashion. If that’s what you want do to with your future self, that is.
there can just as easily be a superintelligence that rewards people predicted to act one way as one that rewards people predicted to act the other.
Yeah, now. But after Omega really, really, appears in front of you, chance of Omega existing is about 1. Chance of No-Mega is still almost non-existent. In this problem, existence of Omega is given. It’s not something you are expecting to encounter now, just as we’re not expecting to encounter eccentric Kavkan billionaires that will give you money for toxicating yourself. The Kavka’s Toxin and the counterfactual mugging present a scenario that is given, and ask you how would you act then.
But you aren’t supposed to be updating… the essence of UDT, I believe, is that your policy should be set NOW, and NEVER UPDATED.
So… either:
You consider the choice of policy based on the prior where you DIDN’T KNOW whether you’d face Nomega or Omega, and NEVER UPDATE IT (this seems obviously wrong to me: why are you using your old prior instead of your current posterior?).
or
You consider the choice of policy based on the prior where you KNOW that you are facing Omega AND that the coin is tails, in which case paying Omega only loses you money.
It doesn’t prevent doing different actions in different circumstances, though. That’s not what “updateless” means. It means that you should act as your past self would have precommitted to doing in your situation. Your probability estimate for “I see Omega” should be significantly greater than “I see Omega, and also Nomega is watching and deciding how to act”, so your decision should be mostly determined by Omega, not Nomega. (The Metanomega also applies—there’s a roughly equal chance of Metanomega or Nomega waiting and watching. [Metanomega = Nomega reversed; gives payoff iff predicts you paying.])
I see where I went wrong. I assumed that the impact of one’s response to Omega is limited to the number of worlds in which Omega exists. That is, my reasoning is invalid if (“what I do in scenario X” is meaningful and affects the world even if scenario X never happens). In other words, when one is being counterfactually modeled, which is exactly the topic of discussion.
Yes, that there can just as easily be a superintelligence that rewards people predicted to act one way as one that rewards people predicted to act the other. Which precommitment is most rational depends depends on the which type you expect to encounter.
I don’t expect to encounter either, and on the other hand I can’t rule out fallible human analogues of either. So for now I’m not precommitting either way.
You don’t precommit to “give away the $100, to anyone who asks”. You precommit to give away the $100 in exactly the situation I described. Or, generalizing such precommitments, you just compute your decisions on the spot, in a reflectively consistent fashion. If that’s what you want do to with your future self, that is.
Yeah, now. But after Omega really, really, appears in front of you, chance of Omega existing is about 1. Chance of No-Mega is still almost non-existent. In this problem, existence of Omega is given. It’s not something you are expecting to encounter now, just as we’re not expecting to encounter eccentric Kavkan billionaires that will give you money for toxicating yourself. The Kavka’s Toxin and the counterfactual mugging present a scenario that is given, and ask you how would you act then.
But you aren’t supposed to be updating… the essence of UDT, I believe, is that your policy should be set NOW, and NEVER UPDATED.
So… either:
You consider the choice of policy based on the prior where you DIDN’T KNOW whether you’d face Nomega or Omega, and NEVER UPDATE IT (this seems obviously wrong to me: why are you using your old prior instead of your current posterior?). or
You consider the choice of policy based on the prior where you KNOW that you are facing Omega AND that the coin is tails, in which case paying Omega only loses you money.
It doesn’t prevent doing different actions in different circumstances, though. That’s not what “updateless” means. It means that you should act as your past self would have precommitted to doing in your situation. Your probability estimate for “I see Omega” should be significantly greater than “I see Omega, and also Nomega is watching and deciding how to act”, so your decision should be mostly determined by Omega, not Nomega. (The Metanomega also applies—there’s a roughly equal chance of Metanomega or Nomega waiting and watching. [Metanomega = Nomega reversed; gives payoff iff predicts you paying.])
I see where I went wrong. I assumed that the impact of one’s response to Omega is limited to the number of worlds in which Omega exists. That is, my reasoning is invalid if (“what I do in scenario X” is meaningful and affects the world even if scenario X never happens). In other words, when one is being counterfactually modeled, which is exactly the topic of discussion.