I’m not sure it’s really more counterintuitive that (known) ability to predict can be a disadvantage than it is that (known) having more options can be a disadvantage.
In this case, predictability is an advantage because it allows you to make binding commitments; in other words, to visibly eliminate options that would otherwise be available to you. And (see, e.g., Schelling) the ability to visibly eliminate some of your own options is very often valuable, because those options might be ones whose possibility gives the other player reason to do something that would be bad for you.
In this case, A’s predictability effectively takes the possibility that A might cooperate out of the picture for B, which means that B no longer has reason to defect.
(The examples in Schelling, IIRC, tend to be of an opposite kind, more like the PD, where the ability to assure the other player that you won’t defect is advantageous for both parties.)
I’m not sure it’s really more counterintuitive that (known) ability to predict can be a disadvantage than it is that (known) having more options can be a disadvantage.
I wonder if it might be fruitful to think generally about decision theories in terms of their ability to rule out suboptimal decisions, as opposed to their ability to select the optimal decision.
I also wanted you to read something I wrote below:
When described in this way, I am reminded that I would be very interested to see this sort of problem examined in the modal agents framework. I have to flag that I lack a technical understanding of this sort of thing, but it seems like we can imagine the agents as formal systems, with B stronger than A, and A forcing B to prove that A defects by making it provable in A that A defects, and since B is stronger than A, it is also provable in B that A defects.
Also, there are variants with imperfect predictors:
It can be shown that A behaves precisely as an agent would if it cooperates with an arbitrary agent C if P(C predicts A defects | A defects) is less than 0.5, is indifferent if that probability equals 0.5, and defects if that probability is greater than 0.5.
Suppose that B’s predictive accuracy is greater than 50 percent. Then the expected utility of A defecting is 2p + 0(1 - p), and the expected utility of A cooperating is 1p + 1(1 - p), and the expected utility of defection is greater than that of cooperation. Plug in numbers if you need to. There are similar proofs that if B’s predictions are random, then A is indifferent, and if B’s predictive ability is less than 50 percent, then A cooperates.
The human did something functionally equivalent to making binding commitments.
it’s UDP that is predictable
Nope, TCP :-).
(If you meant UDT or something of the kind: the claim being made in the OP is that a good implementation of UDT will be very good at predicting, not that it will be very predictable.)
It isn’t making a move that I suggested was equivalent to making a binding commitment. (In this case, it’s working out one’s best strategy, in the presence of a perfect predictor.) It’s equivalent in the sense that both have the effect of narrowing the options the other player thinks you might take. That’s not a good notion of equivalence in all contexts, but I think it is here; the impact on the game is the same.
Yes, there are situations in which UDT-as-understood-by-the-OP produces predictable results. That doesn’t mean that UDT (as understood etc.) Is consistently predictable, and it remains the case that the OP explicitly characterized the UDT-using agent as a superhumanly effective predictor.
I’m not sure it’s really more counterintuitive that (known) ability to predict can be a disadvantage than it is that (known) having more options can be a disadvantage.
In this case, predictability is an advantage because it allows you to make binding commitments; in other words, to visibly eliminate options that would otherwise be available to you. And (see, e.g., Schelling) the ability to visibly eliminate some of your own options is very often valuable, because those options might be ones whose possibility gives the other player reason to do something that would be bad for you.
In this case, A’s predictability effectively takes the possibility that A might cooperate out of the picture for B, which means that B no longer has reason to defect.
(The examples in Schelling, IIRC, tend to be of an opposite kind, more like the PD, where the ability to assure the other player that you won’t defect is advantageous for both parties.)
I wonder if it might be fruitful to think generally about decision theories in terms of their ability to rule out suboptimal decisions, as opposed to their ability to select the optimal decision.
I also wanted you to read something I wrote below:
Also, there are variants with imperfect predictors:
In this case the human did not make any binding commitments.
Not to mention that it’s UDP that is predictable.
The human did something functionally equivalent to making binding commitments.
Nope, TCP :-).
(If you meant UDT or something of the kind: the claim being made in the OP is that a good implementation of UDT will be very good at predicting, not that it will be very predictable.)
Unless you want to screw around with terminology, making a move in a game is not “making a binding commitment”. It’s making a move.
Let’s look at the OP:
This is predicting the opponent’s response. Since UDT (according to the OP) does write down 1, the prediction is accurate.
UDT looks very predictable to me in this case.
It isn’t making a move that I suggested was equivalent to making a binding commitment. (In this case, it’s working out one’s best strategy, in the presence of a perfect predictor.) It’s equivalent in the sense that both have the effect of narrowing the options the other player thinks you might take. That’s not a good notion of equivalence in all contexts, but I think it is here; the impact on the game is the same.
Yes, there are situations in which UDT-as-understood-by-the-OP produces predictable results. That doesn’t mean that UDT (as understood etc.) Is consistently predictable, and it remains the case that the OP explicitly characterized the UDT-using agent as a superhumanly effective predictor.