The reason we shift probability weight away from the deceptive Omega is that, in the original problem, we are told that we believe Omega to be non-deceptive. The reasoning goes like this: If it looks like Omega and talks like Omega, then it might be Omega or Omega . But if it were Omega* , then it would be deceiving us, so it’s most probably Omega.
In the original problem, we have no reason to believe that No-mega and friends are non-deceptive.
(But if we did, then yes, the dual of a non-deceptive agent would be deceptive, and so have lower prior probability. This would be a different problem, but it would still have a symmetry: We would have to define a different notion of dual, where the dual of an agent has the reversed behavior and also reverses its claims about its own behavior.
What would Omega* be in that case? It would not claim to be Omega. It would truthfully tell you that if it predicted you would not give it $5 on tails, then it would flip a coin and give you $100 on heads; and otherwise it would not give you anything. This has no bearing on your decision in the Omega problem.)
By your definitions, Omega would condition its decision on you being counterfactually muggable by the original Omega, not on you giving money to Omega itself. Or am I losing the plot again? This notion of “duality” seems to be getting more and more complex.
“Duality” has become more complex because we’re now talking about a more complex problem — a version of Counterfactual Mugging where you believe that all superintelligent agents are trustworthy. The old version of duality suffices for the ordinary Counterfactual Mugging problem.
My thesis is that there’s always a symmetry in the space of black swans like No-mega.
In the case currently under consideration, I’m assuming Omega’s spiel goes something like “I just flipped a coin. If it had been heads, I would have predicted what you would do if I had approached you and given my spiel....” Notice the use of first-person pronouns. Omega* would have almost the same spiel verbatim, also using first-person pronouns, and make no reference to Omega. And, being non-deceptive, it would behave the way it says it does. So it wouldn’t condition on your being muggable by Omega.
You could object to this by claiming that Omega actually says “I am Omega. If Omega had come up to you and said....”, in which case I can come up with a third notion of duality.
If Omega* makes no reference to the original Omega, I don’t understand why they have “opposite behavior with respect to my status as being counterfactually-muggable” (by the original Omega), which was your reason for inventing “duality” in the first place. I apologize, but at this point it’s unclear to me that you actually have a proof of anything. Maybe we can take this discussion to email?
The reason we shift probability weight away from the deceptive Omega is that, in the original problem, we are told that we believe Omega to be non-deceptive. The reasoning goes like this: If it looks like Omega and talks like Omega, then it might be Omega or Omega . But if it were Omega* , then it would be deceiving us, so it’s most probably Omega.
In the original problem, we have no reason to believe that No-mega and friends are non-deceptive.
(But if we did, then yes, the dual of a non-deceptive agent would be deceptive, and so have lower prior probability. This would be a different problem, but it would still have a symmetry: We would have to define a different notion of dual, where the dual of an agent has the reversed behavior and also reverses its claims about its own behavior.
What would Omega* be in that case? It would not claim to be Omega. It would truthfully tell you that if it predicted you would not give it $5 on tails, then it would flip a coin and give you $100 on heads; and otherwise it would not give you anything. This has no bearing on your decision in the Omega problem.)
Edit: Formatting.
By your definitions, Omega would condition its decision on you being counterfactually muggable by the original Omega, not on you giving money to Omega itself. Or am I losing the plot again? This notion of “duality” seems to be getting more and more complex.
“Duality” has become more complex because we’re now talking about a more complex problem — a version of Counterfactual Mugging where you believe that all superintelligent agents are trustworthy. The old version of duality suffices for the ordinary Counterfactual Mugging problem.
My thesis is that there’s always a symmetry in the space of black swans like No-mega.
In the case currently under consideration, I’m assuming Omega’s spiel goes something like “I just flipped a coin. If it had been heads, I would have predicted what you would do if I had approached you and given my spiel....” Notice the use of first-person pronouns. Omega* would have almost the same spiel verbatim, also using first-person pronouns, and make no reference to Omega. And, being non-deceptive, it would behave the way it says it does. So it wouldn’t condition on your being muggable by Omega.
You could object to this by claiming that Omega actually says “I am Omega. If Omega had come up to you and said....”, in which case I can come up with a third notion of duality.
If Omega* makes no reference to the original Omega, I don’t understand why they have “opposite behavior with respect to my status as being counterfactually-muggable” (by the original Omega), which was your reason for inventing “duality” in the first place. I apologize, but at this point it’s unclear to me that you actually have a proof of anything. Maybe we can take this discussion to email?