From this, they trivially conclude that EDT will have higher stakes than CDT: if there are more Good Twins (Evil Twins), EDT will recommend one-boxing (two-boxing) very strongly, since this will provide evidence to you about many agents doing the same. But I’m not satisfied with this answer, because if you don’t know whether more Good Twins or Evil Twins exist, you won’t be obtaining that evidence (upon taking the decision)!
I don’t think this is a situation of evidential symmetry which would warrant a uniform distribution (i.e. you can’t just say that “you don’t know”). (Moreover, there does not seem to be an overwhelmingly natural partition of the state space in this particular case, which arguably makes the Principle of Indifference inapplicable regardless—see section 3 of Greaves [2016].)
One weak piece of evidence is e.g. provided by the mediocrity principle: since I know for sure that there exists at least one agent who has my preferences and makes decisions in the way I do (me!)—and I don’t know the opposite for sure—I should expect there to be more Good Twins than Evil Twins.
Moreover, I should probably expect there in general to be some correlation between decision theory and values, meaning that my (decision-theoretic) twins are by my lights more likely to be Good than Evil.
As it turns out, you’re right! Yesterday I discussed this issue with Caspar Oesterheld (one of the authors). Indeed, his answer to this objection is that they believe there probably are more positively than negatively correlated agents. Some arguments for that are evolutionary pressures and the correlation between decision theory and values you mention. In this post, I was implicitly relying on digital minds being crazy enough as for a big fraction of them to be negatively correlated to us. This could plausibly be the case in extortion/malevolent actors scenarios, but I don’t have any arguments for that being probable enough.
In fact, I had already come up with a different objection to my argument. And the concept of negatively correlated agents is generally problematic for other reasons. I’ll write another post presenting these and other considerations when I have the time (probably the end of this month). I’ll also go over Greaves [2016], thank you for that resource!
I don’t think this is a situation of evidential symmetry which would warrant a uniform distribution (i.e. you can’t just say that “you don’t know”). (Moreover, there does not seem to be an overwhelmingly natural partition of the state space in this particular case, which arguably makes the Principle of Indifference inapplicable regardless—see section 3 of Greaves [2016].)
One weak piece of evidence is e.g. provided by the mediocrity principle: since I know for sure that there exists at least one agent who has my preferences and makes decisions in the way I do (me!)—and I don’t know the opposite for sure—I should expect there to be more Good Twins than Evil Twins.
Moreover, I should probably expect there in general to be some correlation between decision theory and values, meaning that my (decision-theoretic) twins are by my lights more likely to be Good than Evil.
Thank you for your comment, Sylvester!
As it turns out, you’re right! Yesterday I discussed this issue with Caspar Oesterheld (one of the authors). Indeed, his answer to this objection is that they believe there probably are more positively than negatively correlated agents. Some arguments for that are evolutionary pressures and the correlation between decision theory and values you mention. In this post, I was implicitly relying on digital minds being crazy enough as for a big fraction of them to be negatively correlated to us. This could plausibly be the case in extortion/malevolent actors scenarios, but I don’t have any arguments for that being probable enough.
In fact, I had already come up with a different objection to my argument. And the concept of negatively correlated agents is generally problematic for other reasons. I’ll write another post presenting these and other considerations when I have the time (probably the end of this month). I’ll also go over Greaves [2016], thank you for that resource!
Ah, nice. I was just about to recommend sections 2.6.2 and 3 of Multiverse-wide Cooperation via Correlated Decision Making by Caspar.
Nice, thank you! I will delve into that one as well when I have the time :-)