I realize that this seems to be a common view, but I can’t even begin to imagine how intelligent rational people who have given the matter some thought can possibly think that they are going to be one and only one particular future self even when other future selves exist. If your future selves A and B will both exist, what could possibly be the difference between being only future self A and being only future self B? No one seems to imagine a silver thread or a magical pixie dust trail connecting their present self and a particular future self. Is this supposed to be one of those mysterious “first person facts”? How? Your current self and all of your future selves have exactly the same experiences in either case, unless you expect something to break that symmetry. What would that be?
I’m expressing incomprehension, not disapproval. I’m genuinely puzzled. If I were trying to express disapproval I would have phrased the pixie dust sentence with “It’s like they think there is a … or … or something.”
Am I not allowed to use the the phrase “can’t even begin to imagine” even when I spent the rest of the comment trying to imagine and failing utterly?
In my experience with colloquial English—mostly in the Pacific Northwest, Hawaii, and Florida—”I can’t even begin to imagine [justification for other person’s behavior or belief]” expresses disapproval. Specifically, it expresses the belief that the behavior or belief is not justified, and that the person under discussion is either disingenuous or working with an unusually flawed epistemic strategy.
The phrase may have different connotations for you, but if I were trying to express incomprehension safely I’d choose a different phrase. Maybe something like “Although seemingly intelligent, rational people disagree, anticipating being one and only one particular future self when other future selves exist sounds incoherent.”
I don’t see how calling it incoherent is any less disapproving. And I expect the problem is not the particular phase I used, but the level of incomprehension expressed, otherwise you would have suggested some actual expression of incomprehension, wouldn’t you? What I want to express is that I have no explanation for the presence of the apparent belief and unsuccessfully searched for possible justifications without finding anything that looked promising at any stage. Calling it incoherent would imply that I expect a psychological explanation instead of a justification to be the cause of the belief. I don’t have that expectation.
I meant for my suggestion to denotationally express the same thing as what you said. But the connotations of “I can’t even begin to imagine...” are different from the connotations of ”...sounds incoherent.”
In contexts more familiar to you, ”...sounds incoherent” may express more disapproval; but in the colloquial English I’m familiar with, it’s the more neutral phrase.
This is one of those things that everyone knows explicitly but is probably still worth a post with lots of good examples for new recruits. (Unless it’s already been covered.)
In general I don’t think the sequences covered a lot of failure modes caused by feeling morally or status-ly justified or self-righteous.
I agree probably nothing sets apart particular copies as your future. But it shouldn’t matter to questions like this. You should be able to conceptualise an arbitrary set of things as ‘you’ or as whatever you want to call it, even where there is no useful physical distinction to be made, and still expect probability theory to work.
If your future selves A and B will both exist, what could possibly be the difference between being only future self A and being only future self B?
I know that I am me and you are not me, because (for example) when I want to pick up the pencil on the table between us my arm moves and yours doesn’t.
Future self A can perform that experiment to determine that he is not also future self B. B can also do that with respect to A.
Thus, the anticipation of being both future self A and future self B does not correlate to any particular experience anyone is going to have.
I think you misunderstand me. Future you A is future you A, and not future you B, and likewise future you B is not future you A. No particular you will ever experience being A and B at the same time. Future you A and B both remember being current you, but not being each other. I completely agree with all that. What I don’t understand is how either A or B is supposed to be you in the sense of being the same person as current you while the other is not?
I’m not claiming that A and B have to consider each other to be the same person. That would be a possibility, but they/you could also treat being the same person as non-transitive so each is the same person as current you (C), but they aren’t the same person to each other, or A, B and C could consider themselves three different persons. The only thing that doesn’t make sense to me is C going on to be either A or B, as determined by random chance (??? where would that randomness be happening?) or … something? I don’t even sufficiently understand how this is supposed to work to properly describe it.
What I don’t understand is how either A or B is supposed to be you in the sense of being the same person as current you while the other is not?
A will probably call A the real you, and B will probably call B the real you. Other people might find them both the same as the current you, but might take sides on the labeling issue later if A or B does something they like or don’t like. It’d surely be most useful to call both A and B “the same person as current you” in the beginning, at least, because they’d both be extremely similar to the current you. A might change more than B as time goes on, leading some to prefer identifying B as the “real” you (possibly right away, to dissipate the weirdness of it all), but it’s all a matter of preference in labels. After all, even now the you that is reading this post is not the same as the you of 5 minutes ago. English simply isn’t well-equipped to deal with the situation where a person can have multiple future selves (at least, not yet).
William T. Riker was copied and his copy (hereafter “Thomas Riker”) was abandoned all alone on a planet. Will Riker had friends and a career history since the copy, so for convenience Thomas Riker took the name “Thomas” and continued his old career where he left off.
I agree, but as you allow, your (future) specific identity amongst identical copies matters very much when symmetry is broken, e.g. one copy is to be tortured and the rest pleasured. It matters to me (my experience in the moment) even with some inevitable future destructive merge in all my copies’ future, just as it matters to me what I experience now in spite of the expectation of my permanent death within 100 years.
I agree, but as you allow, your (future) specific identity amongst identical copies matters very much when symmetry is broken, e.g. one copy is to be tortured and the rest pleasured.
I’m not sure I understand you. Obviously it matters to your future self A whether A is tortured or pleasured. And also to your current self whether there is a future self A that will be tortured. Do you think that, given that your future self A is tortured and your future self B pleasured, there is an additional fact as to whether you will be tortured or pleasured? I don’t. And I don’t see the relevance of the rest of your post to my point either.
If I see myselves at different points of time as being in collusion as to how to make all of us better off, which has been a viewpoint I’ve seen taken recently, then there is some agreement between a set of sufficiently-similar agents.
I could view the terms of that agreement as “me” and then the question becomes “what do the terms of the agreement that different sufficiently-similar instances of me serve under say about this situation.”
In which case “I” want to come up with a way of deciding, for example, how much pleasure I require per unit of torture, etc. But certainly the question “Am I being tortured or pleasured” doesn’t exactly carry over.
I thought I disagreed with you but then I showed my work and it turns out I agree with you.
If I see myselves at different points of time as being in collusion as to how to make all of us better off, which has been a viewpoint I’ve seen taken recently, then there is some agreement between a set of sufficiently-similar agents.
If this is too easy, a way to make it more fun is to do the same thing but with parts of you and coalitions of parts of you, gene/meme-eye view of evolution style. Thinking about whether there’s an important metaphysical or decision theoretic sense in which an algorithm is ‘yours’ or ‘mine’ from this perspective, while seeing if it continues to add up to normality, can lead to more fun still. And if that’s still not fun enough you can get really good at the kinds of meditation that supposedly let you intuitively grok all of this nonsense and notice the subtleties from the inside! Maybe. :D
If you’re searching for how I disagree with you, I don’t (I thought I made that clear with “as you allow”). At first you were talking about a perfectly symmetrical situation; I was interested in a less symmetrical one.
Is there an additional fact? No, and at first I was tempted to think that it matters how if there’s a continuous experience vs. a discontinuity where a scanned copy is woken up, (i.e. the original isn’t destroyed, so I might care more, as the original, about what fate lies in store for it). But I think that difference doesn’t even matter to me, assuming perfect copying, of course.
To indulge in another shift: maybe I’ll create slave copies of myself one day. I certainly won’t be allocating an even share of my resources to them (they’ll hate me for it) :)
Do you think that, given that your future self A is tortured and your future self B pleasured, there is an additional fact as to whether you will be tortured or pleasured?
Indeed, there is that additional fact.
You personally don’t know that fact; it is stipulated that until the crucial future divergence, the two selves’ experiences are identical (to them). That’s part of the setup. But Omega, who is looking on, sees two physically separate bodies, and he can label them A and B and keep track of which is which before their experiences begin to diverge.
The information about who is who exists, as part of the state of the universe. We only ignore this because we’re performing thought experiments. But suppose you really ran this experiment. Suppose that the identical experiences of the two selves involved talking to the experimenters, running through two physically distinct but identical copies of the same conversation. Then, if you didn’t think the experimenters were infallible, you might try to trick them during the conversation, to break the symmetry and reveal to you whether you were the copy who were going to be tortured.
Of course the setup stipulates that the symmetry can’t be broken. But that’s an idealization. Because of this, I think that while the results of the thought experiment are internally consistent, they may not fully apply to any physically realizable situation.
I’m not sure I understand you. Do you think that right now there exist an infinite number of physically separate bodies that collectively make up current you and all have identical experiences (including this conversation), that exactly one of those bodies is the real (?) you, and that every distinct possible future you can be traced back to one particular body or a distinct subset of those physical bodies? If so what is your basis for believing that? And is this true for any possible mechanism that could produce copies, or do you refuse to acknowledge copies that are produced in a way that doesn’t preserve this quality as possible yous?
Do you think that right now there exist an infinite number of physically separate bodies that collectively make up current you
No, I don’t think it’s normally the case (I presume you’re referring to quantum branches?).
But the Anthropic Trilemma scenario we’re discussing explicitly postulates this: that physically separate but identical copies of my body will be constructed, have identical experiences for a while, and then diverge.
If this scenario is actually implemented, then it will be necessarily true that (given complete knowledge of the universe) every distinct future body can be traced back to a past body at a point before the experiences diverge.
This would not be true, though, in Eliezer’s other thought experiment about persons implemented as 2D circuits that can be split along their thickness.
But in the scenario here, if there are two bodies and one is going to be tortured tomorrow, there can be a fact of the matter today about which one it is, even if the bodies themselves don’t know it. Even though their presents are identical (to the limit that they can experience), the fact that their futures are going to be different can make it necessary to talk about two separate persons existing all along.
That’s not what I was talking about though, I was talking about the perspective of the root. Obviously once you already are in a branch there is a fact as to whether you are in that branch, even if you can’t tell yourself.
But the Anthropic Trilemma scenario we’re discussing explicitly postulates this: that physically separate but identical copies of my body will be constructed, have identical experiences for a while, and then diverge.
Not as far as I can tell. The first experience after being copied is either winning the lottery or not.
I realize that this seems to be a common view, but I can’t even begin to imagine how intelligent rational people who have given the matter some thought can possibly think that they are going to be one and only one particular future self even when other future selves exist. If your future selves A and B will both exist, what could possibly be the difference between being only future self A and being only future self B? No one seems to imagine a silver thread or a magical pixie dust trail connecting their present self and a particular future self. Is this supposed to be one of those mysterious “first person facts”? How? Your current self and all of your future selves have exactly the same experiences in either case, unless you expect something to break that symmetry. What would that be?
Please don’t express disapproval with incomprehension, it’s an unhealthy epistemic practice.
I’m expressing incomprehension, not disapproval. I’m genuinely puzzled. If I were trying to express disapproval I would have phrased the pixie dust sentence with “It’s like they think there is a … or … or something.”
Am I not allowed to use the the phrase “can’t even begin to imagine” even when I spent the rest of the comment trying to imagine and failing utterly?
In my experience with colloquial English—mostly in the Pacific Northwest, Hawaii, and Florida—”I can’t even begin to imagine [justification for other person’s behavior or belief]” expresses disapproval. Specifically, it expresses the belief that the behavior or belief is not justified, and that the person under discussion is either disingenuous or working with an unusually flawed epistemic strategy.
The phrase may have different connotations for you, but if I were trying to express incomprehension safely I’d choose a different phrase. Maybe something like “Although seemingly intelligent, rational people disagree, anticipating being one and only one particular future self when other future selves exist sounds incoherent.”
I don’t see how calling it incoherent is any less disapproving. And I expect the problem is not the particular phase I used, but the level of incomprehension expressed, otherwise you would have suggested some actual expression of incomprehension, wouldn’t you? What I want to express is that I have no explanation for the presence of the apparent belief and unsuccessfully searched for possible justifications without finding anything that looked promising at any stage. Calling it incoherent would imply that I expect a psychological explanation instead of a justification to be the cause of the belief. I don’t have that expectation.
I meant for my suggestion to denotationally express the same thing as what you said. But the connotations of “I can’t even begin to imagine...” are different from the connotations of ”...sounds incoherent.”
In contexts more familiar to you, ”...sounds incoherent” may express more disapproval; but in the colloquial English I’m familiar with, it’s the more neutral phrase.
This is one of those things that everyone knows explicitly but is probably still worth a post with lots of good examples for new recruits. (Unless it’s already been covered.)
In general I don’t think the sequences covered a lot of failure modes caused by feeling morally or status-ly justified or self-righteous.
Can’t remember the name off the top of my head, but think Eliezer has this one covered.
I agree probably nothing sets apart particular copies as your future. But it shouldn’t matter to questions like this. You should be able to conceptualise an arbitrary set of things as ‘you’ or as whatever you want to call it, even where there is no useful physical distinction to be made, and still expect probability theory to work.
Probability theory works on whatever probability spaces you define. This fact doesn’t justify any particular specification of a probability space.
I know that I am me and you are not me, because (for example) when I want to pick up the pencil on the table between us my arm moves and yours doesn’t.
Future self A can perform that experiment to determine that he is not also future self B. B can also do that with respect to A.
Thus, the anticipation of being both future self A and future self B does not correlate to any particular experience anyone is going to have.
I think you misunderstand me. Future you A is future you A, and not future you B, and likewise future you B is not future you A. No particular you will ever experience being A and B at the same time. Future you A and B both remember being current you, but not being each other. I completely agree with all that. What I don’t understand is how either A or B is supposed to be you in the sense of being the same person as current you while the other is not?
I’m not claiming that A and B have to consider each other to be the same person. That would be a possibility, but they/you could also treat being the same person as non-transitive so each is the same person as current you (C), but they aren’t the same person to each other, or A, B and C could consider themselves three different persons. The only thing that doesn’t make sense to me is C going on to be either A or B, as determined by random chance (??? where would that randomness be happening?) or … something? I don’t even sufficiently understand how this is supposed to work to properly describe it.
A will probably call A the real you, and B will probably call B the real you. Other people might find them both the same as the current you, but might take sides on the labeling issue later if A or B does something they like or don’t like. It’d surely be most useful to call both A and B “the same person as current you” in the beginning, at least, because they’d both be extremely similar to the current you. A might change more than B as time goes on, leading some to prefer identifying B as the “real” you (possibly right away, to dissipate the weirdness of it all), but it’s all a matter of preference in labels. After all, even now the you that is reading this post is not the same as the you of 5 minutes ago. English simply isn’t well-equipped to deal with the situation where a person can have multiple future selves (at least, not yet).
Good point. Fictional example:
William T. Riker was copied and his copy (hereafter “Thomas Riker”) was abandoned all alone on a planet. Will Riker had friends and a career history since the copy, so for convenience Thomas Riker took the name “Thomas” and continued his old career where he left off.
I agree, but as you allow, your (future) specific identity amongst identical copies matters very much when symmetry is broken, e.g. one copy is to be tortured and the rest pleasured. It matters to me (my experience in the moment) even with some inevitable future destructive merge in all my copies’ future, just as it matters to me what I experience now in spite of the expectation of my permanent death within 100 years.
I’m not sure I understand you. Obviously it matters to your future self A whether A is tortured or pleasured. And also to your current self whether there is a future self A that will be tortured. Do you think that, given that your future self A is tortured and your future self B pleasured, there is an additional fact as to whether you will be tortured or pleasured? I don’t. And I don’t see the relevance of the rest of your post to my point either.
If I see myselves at different points of time as being in collusion as to how to make all of us better off, which has been a viewpoint I’ve seen taken recently, then there is some agreement between a set of sufficiently-similar agents.
I could view the terms of that agreement as “me” and then the question becomes “what do the terms of the agreement that different sufficiently-similar instances of me serve under say about this situation.”
In which case “I” want to come up with a way of deciding, for example, how much pleasure I require per unit of torture, etc. But certainly the question “Am I being tortured or pleasured” doesn’t exactly carry over.
I thought I disagreed with you but then I showed my work and it turns out I agree with you.
If this is too easy, a way to make it more fun is to do the same thing but with parts of you and coalitions of parts of you, gene/meme-eye view of evolution style. Thinking about whether there’s an important metaphysical or decision theoretic sense in which an algorithm is ‘yours’ or ‘mine’ from this perspective, while seeing if it continues to add up to normality, can lead to more fun still. And if that’s still not fun enough you can get really good at the kinds of meditation that supposedly let you intuitively grok all of this nonsense and notice the subtleties from the inside! Maybe. :D
If you’re searching for how I disagree with you, I don’t (I thought I made that clear with “as you allow”). At first you were talking about a perfectly symmetrical situation; I was interested in a less symmetrical one.
Is there an additional fact? No, and at first I was tempted to think that it matters how if there’s a continuous experience vs. a discontinuity where a scanned copy is woken up, (i.e. the original isn’t destroyed, so I might care more, as the original, about what fate lies in store for it). But I think that difference doesn’t even matter to me, assuming perfect copying, of course.
To indulge in another shift: maybe I’ll create slave copies of myself one day. I certainly won’t be allocating an even share of my resources to them (they’ll hate me for it) :)
Indeed, there is that additional fact.
You personally don’t know that fact; it is stipulated that until the crucial future divergence, the two selves’ experiences are identical (to them). That’s part of the setup. But Omega, who is looking on, sees two physically separate bodies, and he can label them A and B and keep track of which is which before their experiences begin to diverge.
The information about who is who exists, as part of the state of the universe. We only ignore this because we’re performing thought experiments. But suppose you really ran this experiment. Suppose that the identical experiences of the two selves involved talking to the experimenters, running through two physically distinct but identical copies of the same conversation. Then, if you didn’t think the experimenters were infallible, you might try to trick them during the conversation, to break the symmetry and reveal to you whether you were the copy who were going to be tortured.
Of course the setup stipulates that the symmetry can’t be broken. But that’s an idealization. Because of this, I think that while the results of the thought experiment are internally consistent, they may not fully apply to any physically realizable situation.
I’m not sure I understand you. Do you think that right now there exist an infinite number of physically separate bodies that collectively make up current you and all have identical experiences (including this conversation), that exactly one of those bodies is the real (?) you, and that every distinct possible future you can be traced back to one particular body or a distinct subset of those physical bodies? If so what is your basis for believing that? And is this true for any possible mechanism that could produce copies, or do you refuse to acknowledge copies that are produced in a way that doesn’t preserve this quality as possible yous?
No, I don’t think it’s normally the case (I presume you’re referring to quantum branches?).
But the Anthropic Trilemma scenario we’re discussing explicitly postulates this: that physically separate but identical copies of my body will be constructed, have identical experiences for a while, and then diverge.
If this scenario is actually implemented, then it will be necessarily true that (given complete knowledge of the universe) every distinct future body can be traced back to a past body at a point before the experiences diverge.
This would not be true, though, in Eliezer’s other thought experiment about persons implemented as 2D circuits that can be split along their thickness.
But in the scenario here, if there are two bodies and one is going to be tortured tomorrow, there can be a fact of the matter today about which one it is, even if the bodies themselves don’t know it. Even though their presents are identical (to the limit that they can experience), the fact that their futures are going to be different can make it necessary to talk about two separate persons existing all along.
That’s not what I was talking about though, I was talking about the perspective of the root. Obviously once you already are in a branch there is a fact as to whether you are in that branch, even if you can’t tell yourself.
Not as far as I can tell. The first experience after being copied is either winning the lottery or not.