Oh, I just realised that my description was terrible. Firstly, I said option b added 50 utility to the person in room 2, instead of room 1. Secondly, only the decision of the person in room 1 matters and it determines the utility everyone else gains.
Any map where the person in room 1 selects room A produces −890 utility total (100-99*10), while any map where the person in room 1 selects B produces 50 in total.
Yet, suppose we only compared A and B and option C didn’t exist. Then you always know that you are the original as you are in the red room and none of the clones are. The only issue I can see with this reasoning is if UDT insists that you care about all the clones, even when you know that you are the original before you’ve made your decision.
Okay, then UDT recommends selecting B regardless of whether C is available. The decision is “self-sacrificing” from the decider’s point of view, but that’s fine. Here’s a simpler example: we make two copies of you, then one of them is asked to pay ten dollars so the other can get a million. Of course agreeing to pay is the right choice! That’s how you’d precommit before the copies got created. And if FAI is faced with that kind of choice, I’m damn sure I want it to pay up.
So you care about all clones, even if they have different experiences/you could never have been them? I only thought you meant that UDT assumed that you cared about copies who “could be you”.
It seems like we could make them semi-clones who use the same decision making algorithm as you, but who have a completely different set of memories so long as it doesn’t affect their process for making the decision. In fact, we could ensure that they have different memories from the first moment of their existence. Why do you care about these semi-clones equally about yourself?
However, you choose option C, in addition to being put in a red room, their memories are replaced so that they have same memories as you.
But suppose option C doesn’t exist, would UDT still endorse choosing option B?
At this point the theory starts to become agnostic, it can take an arbitrary “measure of caring” and give you the best decision according to that. If you’re a UDT agent before the experiment and you care equally about all future clones no matter how much they are tampered with, you choose B. On the other hand, if you have zero caring for clones who were tampered with, you choose A. The cutoff point depends on how much you care for inexact clones. The presence or absence of C still doesn’t matter. Does that make sense?
Let’s see if I’ve got it. So you aggregate using the total or average on a per decision basis (or in UDT 1.1 per observation-action mapping) meaning that individuals who count for one decision may not count in another even if they still exist?
Sorry, I’ve tried looking at the formalisation of UDT, but it isn’t particularly easy to follow. It just assumes that you have a utility function that maps from the execution histories of a set of programs to a real number. It doesn’t say anything about how this should be calculated.
I don’t think it is possible to construct anything simpler, but I can explain in more detail
Suppose you only care about perfect clones. If you select decision C, then Omega has made your semi-clones actual clones, so you should aggregate over all individuals. However, if you select A, they are still only semi-clones so you aggregate over just yourself.
Sure. Then the answer to my question: “individuals who count for one decision may not count in another even if they still exist?” is yes. Agreed?
(Specifically, the semi-clones still exist in A and B, they just haven’t had their memories swapped out in such a way such that they would count as clones)
If you agree, then there isn’t an issue. This test case was designed to create an issue for theories that insist that this ought not to occur.
Why is this important? Because it allows us to create no-win scenarios. Suppose we go back to the genie problem where pressing the button creates semi-clones. If you wish to be pelted by eggs, you know that you are the original, so you regret not wishing for the perfect life. The objection you made before doesn’t hold as you don’t care about the semi-clones. But if you wish for the perfect life, you then know that you are overwhelmingly likely to be a semi-clone, so you regret that decision too.
Yeah, now I see what kind of weirdness you’re trying to point out, and it seems to me that you can recreate it without any clones or predictions or even amnesia. Just choose ten selfish people and put them to sleep. Select one at random, wake him up and ask him to choose between two buttons to press. If he presses button 1, give him a mild electric shock, then the experiment ends and everyone wakes up and goes home. But if he presses button 2, give him a candy bar, wake up the rest of the participants in separate rooms and offer the same choice to each, except this time button 1 leads to nothing and button 2 leads to shock. The setup is known in advance to all participants, and let’s assume that getting shocked is as unpleasant as the candy bar is pleasant.
In this problem UDT says you should press button 1. Yeah, you’d feel kinda regretful having to do that, knowing that it makes you the only person to be offered the choice. You could just press button 2, get a nice candy bar instead of a nasty shock, and screw everyone else! But I still feel that UDT is more likely to be right than some other decision theory telling you to press button 2, given what that leads to.
Perhaps it is, but I think it is worth spending some time investigating this and identifying the advantages and disadvantages of different resolutions.
Hmm, I’m not sure that it works. If you press 1, it doesn’t mean that you’re the first person woken. You need them to be something like semi-clones for that. And the memory trick is only for the Irrelevant Considerations argument. That leaves the prediction element which isn’t strictly necessary, but allows the other agents in your reference class (if you choose perfect life) to exist at the same time the original is making its decision, which makes this result even more surprising.
Agree about the semi-clones part. This is similar to the Prisoner’s Dilemma: if you know that everyone else cooperates (presses button 1), you’re better off defecting (pressing button 2). Usually I prefer to talk about problems where everyone has the same preference over outcomes, because in such problems UDT is a Nash equilibrium. Whereas in problems where people have selfish preferences but cooperate due to symmetry, like this problem or the symmetric Prisoner’s Dilemma, UDT still kinda works but stops being a Nash equilibrium. That’s what I was trying to point out in this post.
Oh, I just realised that my description was terrible. Firstly, I said option b added 50 utility to the person in room 2, instead of room 1. Secondly, only the decision of the person in room 1 matters and it determines the utility everyone else gains.
Any map where the person in room 1 selects room A produces −890 utility total (100-99*10), while any map where the person in room 1 selects B produces 50 in total.
Yet, suppose we only compared A and B and option C didn’t exist. Then you always know that you are the original as you are in the red room and none of the clones are. The only issue I can see with this reasoning is if UDT insists that you care about all the clones, even when you know that you are the original before you’ve made your decision.
Okay, then UDT recommends selecting B regardless of whether C is available. The decision is “self-sacrificing” from the decider’s point of view, but that’s fine. Here’s a simpler example: we make two copies of you, then one of them is asked to pay ten dollars so the other can get a million. Of course agreeing to pay is the right choice! That’s how you’d precommit before the copies got created. And if FAI is faced with that kind of choice, I’m damn sure I want it to pay up.
So you care about all clones, even if they have different experiences/you could never have been them? I only thought you meant that UDT assumed that you cared about copies who “could be you”.
It seems like we could make them semi-clones who use the same decision making algorithm as you, but who have a completely different set of memories so long as it doesn’t affect their process for making the decision. In fact, we could ensure that they have different memories from the first moment of their existence. Why do you care about these semi-clones equally about yourself?
However, you choose option C, in addition to being put in a red room, their memories are replaced so that they have same memories as you.
But suppose option C doesn’t exist, would UDT still endorse choosing option B?
At this point the theory starts to become agnostic, it can take an arbitrary “measure of caring” and give you the best decision according to that. If you’re a UDT agent before the experiment and you care equally about all future clones no matter how much they are tampered with, you choose B. On the other hand, if you have zero caring for clones who were tampered with, you choose A. The cutoff point depends on how much you care for inexact clones. The presence or absence of C still doesn’t matter. Does that make sense?
Let’s see if I’ve got it. So you aggregate using the total or average on a per decision basis (or in UDT 1.1 per observation-action mapping) meaning that individuals who count for one decision may not count in another even if they still exist?
Sorry, I’ve tried looking at the formalisation of UDT, but it isn’t particularly easy to follow. It just assumes that you have a utility function that maps from the execution histories of a set of programs to a real number. It doesn’t say anything about how this should be calculated.
Hmm, not sure I understand the question. Can you make an example problem where “individuals who count for one decision may not count in another”?
I don’t think it is possible to construct anything simpler, but I can explain in more detail
Suppose you only care about perfect clones. If you select decision C, then Omega has made your semi-clones actual clones, so you should aggregate over all individuals. However, if you select A, they are still only semi-clones so you aggregate over just yourself.
Is that a correct UDT analysis?
Hmm, now the problem seems equivalent to this:
A) Get 100 utility
B) Get 50 utility
C) Create many clones and give each −1000 utility
If you’re indifferent to mere existence of clones otherwise, you should choose A. Seems trivial, no?
Sure. Then the answer to my question: “individuals who count for one decision may not count in another even if they still exist?” is yes. Agreed?
(Specifically, the semi-clones still exist in A and B, they just haven’t had their memories swapped out in such a way such that they would count as clones)
If you agree, then there isn’t an issue. This test case was designed to create an issue for theories that insist that this ought not to occur.
Why is this important? Because it allows us to create no-win scenarios. Suppose we go back to the genie problem where pressing the button creates semi-clones. If you wish to be pelted by eggs, you know that you are the original, so you regret not wishing for the perfect life. The objection you made before doesn’t hold as you don’t care about the semi-clones. But if you wish for the perfect life, you then know that you are overwhelmingly likely to be a semi-clone, so you regret that decision too.
Yeah, now I see what kind of weirdness you’re trying to point out, and it seems to me that you can recreate it without any clones or predictions or even amnesia. Just choose ten selfish people and put them to sleep. Select one at random, wake him up and ask him to choose between two buttons to press. If he presses button 1, give him a mild electric shock, then the experiment ends and everyone wakes up and goes home. But if he presses button 2, give him a candy bar, wake up the rest of the participants in separate rooms and offer the same choice to each, except this time button 1 leads to nothing and button 2 leads to shock. The setup is known in advance to all participants, and let’s assume that getting shocked is as unpleasant as the candy bar is pleasant.
In this problem UDT says you should press button 1. Yeah, you’d feel kinda regretful having to do that, knowing that it makes you the only person to be offered the choice. You could just press button 2, get a nice candy bar instead of a nasty shock, and screw everyone else! But I still feel that UDT is more likely to be right than some other decision theory telling you to press button 2, given what that leads to.
Perhaps it is, but I think it is worth spending some time investigating this and identifying the advantages and disadvantages of different resolutions.
Hmm, I’m not sure that it works. If you press 1, it doesn’t mean that you’re the first person woken. You need them to be something like semi-clones for that. And the memory trick is only for the Irrelevant Considerations argument. That leaves the prediction element which isn’t strictly necessary, but allows the other agents in your reference class (if you choose perfect life) to exist at the same time the original is making its decision, which makes this result even more surprising.
Agree about the semi-clones part. This is similar to the Prisoner’s Dilemma: if you know that everyone else cooperates (presses button 1), you’re better off defecting (pressing button 2). Usually I prefer to talk about problems where everyone has the same preference over outcomes, because in such problems UDT is a Nash equilibrium. Whereas in problems where people have selfish preferences but cooperate due to symmetry, like this problem or the symmetric Prisoner’s Dilemma, UDT still kinda works but stops being a Nash equilibrium. That’s what I was trying to point out in this post.