Agreed that in general one will have some uncertainty over whether one’s opponent is the type of algorithm who one boxes / cooperates / whom one wants to cooperate with, etc. It does look like you need to plug these uncertainties into your expected utility calculation, such that you decide to cooperate or defect based on your degree of uncertainty about your opponent.
However, in some cases at least, you don’t need to be Omega-superior to predict whether another agent one-boxes....for example, if you’re facing a clone of yourself; you can just ask yourself what you would do, and you know the answer. There may be some class of algorithms non-identical to you but which are still close enough to you to make this self-reflection increased evidence that your opponent will cooperate if you do.
No, you can’t ask yourself what you’ll do. It’s like a calculator that seeks the answers to the question of “what is 2+2?” in a form “what will I answer to the question “what is 2+2″?”, in which case the answer 57 will be perfectly reasonable.
If you are cooperating with your copy, you only know that the copy will do the same action, which is a restriction on your joint state space. Given this restriction, the expected utility calculation for your actions will return a result different from what other restrictions may force. In this case, you are left only with 2 options: (C,C) and (D,D), of which (C,C) is better.
you’re right. speaking more precisely, by “ask yourself what you would do”, I mean “engage in the act of reflecting, wherein you realize the symmetry between you and your opponent which reduces the decision problem to (C,C) and (D,D), so that you choose (C,C)”, as you’ve outlined above. Note though that even when the reduction is not complete (for example, b/c you’re fighting a similar but inexact clone), there can still be added incentive to cooperate...
Agreed that in general one will have some uncertainty over whether one’s opponent is the type of algorithm who one boxes / cooperates / whom one wants to cooperate with, etc. It does look like you need to plug these uncertainties into your expected utility calculation, such that you decide to cooperate or defect based on your degree of uncertainty about your opponent.
However, in some cases at least, you don’t need to be Omega-superior to predict whether another agent one-boxes....for example, if you’re facing a clone of yourself; you can just ask yourself what you would do, and you know the answer. There may be some class of algorithms non-identical to you but which are still close enough to you to make this self-reflection increased evidence that your opponent will cooperate if you do.
No, you can’t ask yourself what you’ll do. It’s like a calculator that seeks the answers to the question of “what is 2+2?” in a form “what will I answer to the question “what is 2+2″?”, in which case the answer 57 will be perfectly reasonable.
If you are cooperating with your copy, you only know that the copy will do the same action, which is a restriction on your joint state space. Given this restriction, the expected utility calculation for your actions will return a result different from what other restrictions may force. In this case, you are left only with 2 options: (C,C) and (D,D), of which (C,C) is better.
you’re right. speaking more precisely, by “ask yourself what you would do”, I mean “engage in the act of reflecting, wherein you realize the symmetry between you and your opponent which reduces the decision problem to (C,C) and (D,D), so that you choose (C,C)”, as you’ve outlined above. Note though that even when the reduction is not complete (for example, b/c you’re fighting a similar but inexact clone), there can still be added incentive to cooperate...