Hmm. I previously mentioned that in my model of personal identity, our brains include planning machinery that’s based on subjective expectation (“if I do this, what do I expect to experience as a result?”), and that this requires some definition of a “self”, causing our brains to always have a model for continuity of self that they use in predicting the future.
Similarly, in the comments of The Anthropic Trilemma, Eliezer says:
It seems to me that there’s some level on which, even if I say very firmly, “I now resolve to care only about future versions of myself who win the lottery! Only those people are defined as Eliezer Yudkowskys!”, and plan only for futures where I win the lottery, then, come the next day, I wake up, look at the losing numbers, and say, “Damnit! What went wrong? I thought personal continuity was strictly subjective, and I could redefine it however I wanted!”
Translating those notions into the terminology of this post, it would seem like “personal identity” forms an important part of humans’ bridge hypothesis: it is a rule that links some specific entity in the world-model into the agent’s predicted subjective experience. If I believe that there is a personal continuity between me today and me tomorrow, that means that I predict experiencing the things that me-tomorrow will experience, which means that my bridge hypothesis privileges the me of tomorrow over other agents.
I get the feeling that here lies the answer to Eliezer’s question, but I can’t quite put my finger on the exact formulation. Something like “you can alter your world-model or even your model of your own bridge hypothesis, but you can’t alter...” the actual bridge hypothesis? The actual bridge? Something else?
FWIW, I don’t really see what the line you quote adds to the discussion here.
I mean, I believe that preferences for white wine over red or vice-versa are strictly subjective; there’s nothing objectively preferable about one over the other. It doesn’t follow that I can say very firmly “I now resolve to prefer red wine!” and subsequently experience red wine as preferable to white wine. And from this we conclude… nothing much, actually.
Conversely, if Eliezer said “Only people who win the lottery are me!” and the next day the numbers Eliezer picked didn’t win, and when I talked to Eliezer it turned out they genuinely didn’t identify as Eliezer anymore, and their body was going along identifying as someone different… it’s not really clear what we could conclude from that, either.
“Damnit! What went wrong? I thought personal continuity was strictly subjective, and I could redefine it however I wanted!”
Your plan succeeded. Nothing went wrong. By your own admission you have ceased to be EY. In fact, you are an entirely new person who has just this moment come to be alive in a body negligently left behind with obligations by one EY, recently semantically deceased.
Seriously, what Eliezer did there was not “adapting a belief” but “pretending to have adapted a belief”. His claim that, paraphrased, “I define only those people as EY” is simply false as a matter of reporting on his own brain.
I would have to say, “You can’t alter your actual self.” You are going to experience losing the lottery, even if you commit to not doing so. Whether or not you commit to accepting your future self as being yourself, the future you is going to wake up, lose the lottery, and remember being current you. That is a feature of his cognition: he remembers being you, you predict being him. Changing your prediction won’t change his memory, and certainly won’t change the events he experiences in his present.
Well, if subjectivity means “I decide what it is”, then this is tautologically true. If you have a broader definition of subjectivity, then yes, they don’t seem to have much to do with each other. It seems that he was using the first definition, or something similar to it.
When I think of “things that are (defined to be) subjective”, the idea that comes to mind is that of qualia. The perceiver of qualia isn’t in control of them — if I’m experiencing redness, I can’t really choose that it be blue instead. I can say that I’m perceiving blue, but I’d be lying about my own experience.
Fair enough. I am just pointing out the solution to your confusion: You are talking past one another. Words can be wrong, and it is essential to make sure that such things are sorted out properly for an intelligent discussion.
Hmm. I previously mentioned that in my model of personal identity, our brains include planning machinery that’s based on subjective expectation (“if I do this, what do I expect to experience as a result?”), and that this requires some definition of a “self”, causing our brains to always have a model for continuity of self that they use in predicting the future.
Similarly, in the comments of The Anthropic Trilemma, Eliezer says:
Translating those notions into the terminology of this post, it would seem like “personal identity” forms an important part of humans’ bridge hypothesis: it is a rule that links some specific entity in the world-model into the agent’s predicted subjective experience. If I believe that there is a personal continuity between me today and me tomorrow, that means that I predict experiencing the things that me-tomorrow will experience, which means that my bridge hypothesis privileges the me of tomorrow over other agents.
I get the feeling that here lies the answer to Eliezer’s question, but I can’t quite put my finger on the exact formulation. Something like “you can alter your world-model or even your model of your own bridge hypothesis, but you can’t alter...” the actual bridge hypothesis? The actual bridge? Something else?
FWIW, I don’t really see what the line you quote adds to the discussion here.
I mean, I believe that preferences for white wine over red or vice-versa are strictly subjective; there’s nothing objectively preferable about one over the other. It doesn’t follow that I can say very firmly “I now resolve to prefer red wine!” and subsequently experience red wine as preferable to white wine. And from this we conclude… nothing much, actually.
Conversely, if Eliezer said “Only people who win the lottery are me!” and the next day the numbers Eliezer picked didn’t win, and when I talked to Eliezer it turned out they genuinely didn’t identify as Eliezer anymore, and their body was going along identifying as someone different… it’s not really clear what we could conclude from that, either.
-
Your plan succeeded. Nothing went wrong. By your own admission you have ceased to be EY. In fact, you are an entirely new person who has just this moment come to be alive in a body negligently left behind with obligations by one EY, recently semantically deceased.
Seriously, what Eliezer did there was not “adapting a belief” but “pretending to have adapted a belief”. His claim that, paraphrased, “I define only those people as EY” is simply false as a matter of reporting on his own brain.
I would have to say, “You can’t alter your actual self.” You are going to experience losing the lottery, even if you commit to not doing so. Whether or not you commit to accepting your future self as being yourself, the future you is going to wake up, lose the lottery, and remember being current you. That is a feature of his cognition: he remembers being you, you predict being him. Changing your prediction won’t change his memory, and certainly won’t change the events he experiences in his present.
You have to actually go out and rig the lottery.
“You can alter your world-model or even your model of your own bridge hypothesis, but you can’t alter all constraints on types of bridge hypotheses.”
These two claims don’t seem to have much to do with each other.
Well, if subjectivity means “I decide what it is”, then this is tautologically true. If you have a broader definition of subjectivity, then yes, they don’t seem to have much to do with each other. It seems that he was using the first definition, or something similar to it.
When I think of “things that are (defined to be) subjective”, the idea that comes to mind is that of qualia. The perceiver of qualia isn’t in control of them — if I’m experiencing redness, I can’t really choose that it be blue instead. I can say that I’m perceiving blue, but I’d be lying about my own experience.
Fair enough. I am just pointing out the solution to your confusion: You are talking past one another. Words can be wrong, and it is essential to make sure that such things are sorted out properly for an intelligent discussion.