Insofar as I have hope in decision theory leading us to have nice things, it mostly comes via the possibility that a fully-fleshed-out version of UDT would recommend updating “all the way back” to a point where there’s uncertainty about which agent you are. (I haven’t thought about this much and this could be crazy.)
This was surprising to me. For one, that seems like way too much updatelessness. Do you have in mind an agent self-modifying into something like that? If so, when and why? Plausibly this would be after the point of the agent knowing whether it is aligned or not; and I guess I don’t see why there is an incentive for the agent to go updateless with respect to its values at that point in time.[1] At the very least, I think this would not be an instance of all-upside updatelessness.
Secondly, what you are pointing towards seems to be a very specific type of acausal trade. As far as I know, there are three types of acausal trade that are not based on inducing self-locating uncertainty (definitely might be neglecting something here): (1) mutual simulation stuff; (2) ECL; and (3) the thing you describe where agents are being so updateless that they don’t know what they value. (You can of course imagine combinations of these.) So it seems like you are claiming that (3) is more likely and more important than (1) and (2). (Is that right?)
I think I disagree:
The third type seems to only be a possibility under an extreme form of UDT, whilst the other two are possible, albeit in varying degrees, under (updateless or updateful) EDT, TDT, some variants of (updateful or updateless) CDT[2], and a variety of UDTs (including much less extreme ones than the aforementioned variant) and probably a bunch of other theories that I don’t know about or am forgetting.
And I think that the type of updateless you describe seems particularly unlikely, per my first paragraph.
It is not necessarily the case that your expectation of what you will value in the future, when standing behind the veil of ignorance, nicely maps onto the actual distribution of values in the multiverse (meaning the trade you will be doing is going to be limited).
(I guess you can have the third type of trade, and not the first and second one, under standard CDT coupled with the strong updatelessness you describe; which is a point in favour of the claim I think you are making—although this seems fairly weak.)
One argument might be that your decision to go updateless in that way is correlated with the choice of the other agents to go updateless in the same way, and then you get the gains from trade by both being uncertain about what you value. But if you are already sufficiently correlated, and taking this into account for the purposes of decision-making, it is not clear to me why you are not just doing ECL directly.
I’m not claiming this (again, it’s about relative not absolute likelihood).
I’m confused. I was comparing the likelihood of (3) to the likelihood of (1) and (2); i.e. saying something about relative likelihood, no?
I’m not saying this is likely, just that this is the most plausible path I see by which UDT leads to nice things for us.
I meant for my main argument to be directed at the claim of relative likelihood; sorry if that was not clear. So I guess my question is: do you think the updatelessness-based trade you described is the most plausible type of acausal trade out of the three that I listed? As said, ECL and simulation-based trade arguably require much fewer assumptions about decision theory. To get ECL off the ground, for example, you arguably just need your decision theory to cooperate in the Twin PD, and many theories satisfy this criterion.
(And the topic of this post is how decision theory leads us to have nice things, not UDT specifically. Or at least I think it should be; I don’t think one ought to be so confident that UDT/FDT is clearly the “correct” theory [not saying this is what you believe], especially given how underdeveloped it is compared to the alternatives.)
This was surprising to me. For one, that seems like way too much updatelessness. Do you have in mind an agent self-modifying into something like that? If so, when and why? Plausibly this would be after the point of the agent knowing whether it is aligned or not; and I guess I don’t see why there is an incentive for the agent to go updateless with respect to its values at that point in time.[1] At the very least, I think this would not be an instance of all-upside updatelessness.
Secondly, what you are pointing towards seems to be a very specific type of acausal trade. As far as I know, there are three types of acausal trade that are not based on inducing self-locating uncertainty (definitely might be neglecting something here): (1) mutual simulation stuff; (2) ECL; and (3) the thing you describe where agents are being so updateless that they don’t know what they value. (You can of course imagine combinations of these.) So it seems like you are claiming that (3) is more likely and more important than (1) and (2). (Is that right?)
I think I disagree:
The third type seems to only be a possibility under an extreme form of UDT, whilst the other two are possible, albeit in varying degrees, under (updateless or updateful) EDT, TDT, some variants of (updateful or updateless) CDT[2], and a variety of UDTs (including much less extreme ones than the aforementioned variant) and probably a bunch of other theories that I don’t know about or am forgetting.
And I think that the type of updateless you describe seems particularly unlikely, per my first paragraph.
It is not necessarily the case that your expectation of what you will value in the future, when standing behind the veil of ignorance, nicely maps onto the actual distribution of values in the multiverse (meaning the trade you will be doing is going to be limited).
(I guess you can have the third type of trade, and not the first and second one, under standard CDT coupled with the strong updatelessness you describe; which is a point in favour of the claim I think you are making—although this seems fairly weak.)
One argument might be that your decision to go updateless in that way is correlated with the choice of the other agents to go updateless in the same way, and then you get the gains from trade by both being uncertain about what you value. But if you are already sufficiently correlated, and taking this into account for the purposes of decision-making, it is not clear to me why you are not just doing ECL directly.
For example, Spohn’s variant of CDT.
Yepp, I agree. I’m not saying this is likely, just that this is the most plausible path I see by which UDT leads to nice things for us.
I’m not claiming this (again, it’s about relative not absolute likelihood).
I’m confused. I was comparing the likelihood of (3) to the likelihood of (1) and (2); i.e. saying something about relative likelihood, no?
I meant for my main argument to be directed at the claim of relative likelihood; sorry if that was not clear. So I guess my question is: do you think the updatelessness-based trade you described is the most plausible type of acausal trade out of the three that I listed? As said, ECL and simulation-based trade arguably require much fewer assumptions about decision theory. To get ECL off the ground, for example, you arguably just need your decision theory to cooperate in the Twin PD, and many theories satisfy this criterion.
(And the topic of this post is how decision theory leads us to have nice things, not UDT specifically. Or at least I think it should be; I don’t think one ought to be so confident that UDT/FDT is clearly the “correct” theory [not saying this is what you believe], especially given how underdeveloped it is compared to the alternatives.)