Following Nominull and Furcas, I bite the third bullet without qualms for the perfectly ordinary obvious reasons. Once we know how much of what kinds of experiences will occur at different times, there’s nothing left to be confused about. Subjective selfishness is still coherent because you’re not just an arbitrary observer with no distinguishing characteristics at all; you’re a very specific bundle of personality traits, memories, tendencies of thought, and so forth. Subjective selfishness corresponds to only caring about this one highly specific bundle: only caring about whether someone falls off a cliff if this person identifies as such-and-such and has such-and-these specific memories and such-and-those personality traits: however close a correspondence you need to match whatever you define as personal identity.
The popular concepts of altruism and selfishness weren’t designed for people who understand materialism. Once you realize this, you can just recast whatever it was you were already trying to do in terms of preferences over histories of the universe. It all adds up to, &c., &c.
I agree that giving up anticipation does not mean giving up selfishness. But as Dan Armak pointed out there is another reason why you may not want to give up anticipation: you may prefer to keep the qualia of anticipation itself, or more generally do not want to depart too much from the subjective experience of being human.
Eliezer, if you are reading this, why do you not want to give up anticipation? Do you still think it means giving up selfishness? Is it for Dan Armak’s reason? Or something else?
The (only) trouble with this is that it doesn’t answer the question about what probabilities you_0 should assign to various experiences 5 seconds later. Personal identity may not be ontologically fundamental, it may not even be the appropriate sort of thing to be programmed into a utility function—but at the level of our everyday existence (that is, at whatever level we actually do exist), we still have to be able to make plans for “our own” future.
I would say that the ordinarily very useful abstraction of subjective probability breaks down in situations that involve copying and remerging people, and that our intuitive morality breaks down when it has to deal with measure of experience. In the current technological regime, this isn’t a problem at all, because the only branching we do is quantum branching, and there we have this neat correspondence between quantum measure and subjective probability, so you can plan for “your own” future in the ordinary obvious way. How you plan for “your own” future in situations where you expect to be copied and merged depends on the details of your preferences about measure of experience. For myself, I don’t know how I would go about forming such preferences, because I don’t understand consciousness.
In the current technological regime, this isn’t a problem...[the future] depends on the details of your preferences about measure of experience.
Quantum suicide is already a problem in the current regime, if you allow preference over measure.
Splitting and merging adds another problem, but I think it is a factual problem, not an ethical problem. At least, I think that there is a factual problem before you come to the ethical problem, which may be the same as for Born measure.
Following Nominull and Furcas, I bite the third bullet without qualms for the perfectly ordinary obvious reasons. Once we know how much of what kinds of experiences will occur at different times, there’s nothing left to be confused about. Subjective selfishness is still coherent because you’re not just an arbitrary observer with no distinguishing characteristics at all; you’re a very specific bundle of personality traits, memories, tendencies of thought, and so forth. Subjective selfishness corresponds to only caring about this one highly specific bundle: only caring about whether someone falls off a cliff if this person identifies as such-and-such and has such-and-these specific memories and such-and-those personality traits: however close a correspondence you need to match whatever you define as personal identity.
The popular concepts of altruism and selfishness weren’t designed for people who understand materialism. Once you realize this, you can just recast whatever it was you were already trying to do in terms of preferences over histories of the universe. It all adds up to, &c., &c.
I agree that giving up anticipation does not mean giving up selfishness. But as Dan Armak pointed out there is another reason why you may not want to give up anticipation: you may prefer to keep the qualia of anticipation itself, or more generally do not want to depart too much from the subjective experience of being human.
Eliezer, if you are reading this, why do you not want to give up anticipation? Do you still think it means giving up selfishness? Is it for Dan Armak’s reason? Or something else?
The (only) trouble with this is that it doesn’t answer the question about what probabilities you_0 should assign to various experiences 5 seconds later. Personal identity may not be ontologically fundamental, it may not even be the appropriate sort of thing to be programmed into a utility function—but at the level of our everyday existence (that is, at whatever level we actually do exist), we still have to be able to make plans for “our own” future.
I would say that the ordinarily very useful abstraction of subjective probability breaks down in situations that involve copying and remerging people, and that our intuitive morality breaks down when it has to deal with measure of experience. In the current technological regime, this isn’t a problem at all, because the only branching we do is quantum branching, and there we have this neat correspondence between quantum measure and subjective probability, so you can plan for “your own” future in the ordinary obvious way. How you plan for “your own” future in situations where you expect to be copied and merged depends on the details of your preferences about measure of experience. For myself, I don’t know how I would go about forming such preferences, because I don’t understand consciousness.
Quantum suicide is already a problem in the current regime, if you allow preference over measure.
Splitting and merging adds another problem, but I think it is a factual problem, not an ethical problem. At least, I think that there is a factual problem before you come to the ethical problem, which may be the same as for Born measure.