Why average utility of my descendants/copies, instead of total utility? Total utility seems to give better answers. Total utility implies that if copies have better-than-nothing lives, more is better. But that seems right, for roughly the same reason that I don’t want to die in my sleep tonight: it deprives me of good future days. Suppose I learn that I will soon lose long-term (>24 hr) episodic memory, so that every future day will be disconnected from every other, but my life will otherwise be good. Do I still prefer a long life over a one-more-day life? I think yes. But now my days might as well, for all practical and ethical purposes, be lived parallel instead of serially.
With total utility, there is only a very ordinary precommitment problem in Tropical Paradise, provided one important feature. The important feature is that uploaded-me should not be overburdened. Suppose uploaded-me can only afford to make simultaneously-running copies on special occasions, and is reluctant to waste that on this project. That seems reasonable. If uploaded me has to sacrifice 1000 minutes of warm fuzzy feelings to give me one minute of hope now, that’s not worth it. On the other hand, if he only has to do this once—giving me a 50⁄50 hope right now—that may well be worth it.
Let’s make up some numbers. My present wintry blast with no hope of immediate relief, let’s give a utility of zero per minute. Wintry blast with 50⁄50 hope, 6 per minute. Wintry blast with 999/1000 hope, 8 per minute. Tropical paradise, 10 per minute. Summing over all the me and future-me minutes gives the best result with only a single reliving of Winter.
Upload-me makes the sacrifice of 1 minute for basically the same reason Parfit’s hitch-hiker pays his rescuer.
. Suppose I learn that I will soon lose long-term (>24 hr) episodic memory, so that every future day will be disconnected from every other, but my life will otherwise be good. Do I still prefer a long life over a one-more-day life?
Under the model of selfish preferences I use in this post, this is an interesting situation. Suppose that you go to sleep in the same room every night, and every morning you wake up with only your long-term memories (Or your brain is overwritten with the same brain-state every morning or something). Suppose you could give up some tasty candy now for tasty candy every day of your illness. If you eat the candy now, are you robbing yourself of a bunch of future candy, and making a terrible mistake? And yet, every morning a new causal branch of you will wake up, and from their perspective they merely ate their candy a little earlier.
One could even defend not letting yourself get killed off after one day as an altrustic preference rather than a selfish one.
But really this all derived from resolving one single conflict—if there are multiple different conflicts there are multiple solutions. So I’m not really sure—as I hope I sufficiently emphasized, I do not trust this population ethics result.
If you eat the candy now, are you robbing yourself of a bunch of future candy, and making a terrible mistake? And yet, every morning a new causal branch of you will wake up, and from their perspective they merely ate their candy a little earlier.
Cool, this leads me to a new point/question. You’ve defined “selfish” preference in terms of causal flows. I’d like to point out that those flows are not identity-relation-like. Each future branch of me wakes up and sees a one-to-one tradeoff: he doesn’t get candy now, but he got it earlier, so it’s a wash. But those time-slices aren’t the decider, this current one is. And from my perspective now, it’s a many-to-one tradeoff; those future days are all connected to me-now. This is possible because “A is causally connected to B” is intransitive. Isn’t this the correct implication of your view? If not, then what?
Well, the issue is in how one calculates expected utility from a description of the future state of the world. If my current self branches into many causal descendants, and each descendant gets one cookie, there does not appear to be a law of physics that requires me to give that the expected utility of one cookie or many cookies.
It’s absolutely a many to one tradeoff, that just isn’t sufficient to determine how to value it.
However, if one requires that the ancestor and the descendants agree (up to time discounting and selection effects—which are where you value a cookie in 100 years less if you expect to die before then) about the value of a cookie, then that sets a constraint on how to calculate expected utility.
Fair enough. Of course, there’s no law of physics ruling out Future Tuesday Indifference, either. We go by plausibility and elegance. Admittedly, “average the branches” looks about equally plausible and elegant to “sum the branches”, but I think the former becomes implausible when we look at cases where some of the branches are very short-lived.
Requiring that the ancestor and descendants agree is contrary to the spirit of allowing selfish preferences, I think, in the sense of “selfish” that you’ve defined. If Methuselah is selfish, Methuselah(1000AD) values the experience of Methuselah(900AD), who values the experience of Methuselah(800AD), but M1000 doesn’t value the experience of M800.
I think the former becomes implausible when we look at cases where some of the branches are very short-lived.
As the caveat goes, “The copies have to be people who you would actually like to be.” Dying quickly seems like it would really put a damper on the expected utility of being a copy. (Mathematically, the relevant utility here is a time-integral)
I don’t see why your claims about Methuselah follow, but I do agree that under this model, agents don’t care about their past self—they just do what causes them to have high expected utility. Strictly, this is possible independent of whether descendants and ancestors agree or disagree. But if self-modification is possible, such conflicting selfish preferences would get modified away into nonconflicting selfless preferences.
Dying quickly seems like it would really put a damper on the expected utility of being a copy.
Not if the copy doesn’t anticipate dying. Perhaps all the copies go thru a brief dim-witted phase of warm happiness (and the original expects this), in which all they can think is “yup warm and happy, just like I expected”, followed by some copies dying and others recovering full intellect and living. Any of those copies is someone I’d “like to be” in the better-than-nothing sense. Is the caveat “like to be” a stronger sense?
I’m confused—if agents don’t value their past self, in what sense do they agree or disagree with what the past-self was valuing? In any case, please reverse the order of the Methuselah valuing of time-slices.
Edit: Let me elaborate a story to motivate my some-copies-dying posit. I want to show that I’m not just “gaming the system,” as you were concerned to avoid using your caveat.
I’m in one spaceship of a fleet of fast unarmed robotic spaceships. As I feared but planned for, an enemy fleet shows up. This spaceship will be destroyed, but I can make copies of myself in one to all of the many other ships. Each copy will spend 10 warm-and-fuzzy dim-witted minutes reviving from their construction. The space battle will last 5 minutes. The spaceship at the farthest remove from the enemy has about a 10% chance of survival. The next-farthest has a 9 point something percent chance—and so on. The enemy uses an indeterministic algorithm to chase/target ships, so these probabilities are almost independent. If I copy to all the ships in the fleet, I have a very high probability of survival. But the maximum average expected utility is gotten by copying to just one ship.
Why average utility of my descendants/copies, instead of total utility? Total utility seems to give better answers. Total utility implies that if copies have better-than-nothing lives, more is better. But that seems right, for roughly the same reason that I don’t want to die in my sleep tonight: it deprives me of good future days. Suppose I learn that I will soon lose long-term (>24 hr) episodic memory, so that every future day will be disconnected from every other, but my life will otherwise be good. Do I still prefer a long life over a one-more-day life? I think yes. But now my days might as well, for all practical and ethical purposes, be lived parallel instead of serially.
With total utility, there is only a very ordinary precommitment problem in Tropical Paradise, provided one important feature. The important feature is that uploaded-me should not be overburdened. Suppose uploaded-me can only afford to make simultaneously-running copies on special occasions, and is reluctant to waste that on this project. That seems reasonable. If uploaded me has to sacrifice 1000 minutes of warm fuzzy feelings to give me one minute of hope now, that’s not worth it. On the other hand, if he only has to do this once—giving me a 50⁄50 hope right now—that may well be worth it.
Let’s make up some numbers. My present wintry blast with no hope of immediate relief, let’s give a utility of zero per minute. Wintry blast with 50⁄50 hope, 6 per minute. Wintry blast with 999/1000 hope, 8 per minute. Tropical paradise, 10 per minute. Summing over all the me and future-me minutes gives the best result with only a single reliving of Winter.
Upload-me makes the sacrifice of 1 minute for basically the same reason Parfit’s hitch-hiker pays his rescuer.
Under the model of selfish preferences I use in this post, this is an interesting situation. Suppose that you go to sleep in the same room every night, and every morning you wake up with only your long-term memories (Or your brain is overwritten with the same brain-state every morning or something). Suppose you could give up some tasty candy now for tasty candy every day of your illness. If you eat the candy now, are you robbing yourself of a bunch of future candy, and making a terrible mistake? And yet, every morning a new causal branch of you will wake up, and from their perspective they merely ate their candy a little earlier.
One could even defend not letting yourself get killed off after one day as an altrustic preference rather than a selfish one.
But really this all derived from resolving one single conflict—if there are multiple different conflicts there are multiple solutions. So I’m not really sure—as I hope I sufficiently emphasized, I do not trust this population ethics result.
Cool, this leads me to a new point/question. You’ve defined “selfish” preference in terms of causal flows. I’d like to point out that those flows are not identity-relation-like. Each future branch of me wakes up and sees a one-to-one tradeoff: he doesn’t get candy now, but he got it earlier, so it’s a wash. But those time-slices aren’t the decider, this current one is. And from my perspective now, it’s a many-to-one tradeoff; those future days are all connected to me-now. This is possible because “A is causally connected to B” is intransitive. Isn’t this the correct implication of your view? If not, then what?
Well, the issue is in how one calculates expected utility from a description of the future state of the world. If my current self branches into many causal descendants, and each descendant gets one cookie, there does not appear to be a law of physics that requires me to give that the expected utility of one cookie or many cookies.
It’s absolutely a many to one tradeoff, that just isn’t sufficient to determine how to value it.
However, if one requires that the ancestor and the descendants agree (up to time discounting and selection effects—which are where you value a cookie in 100 years less if you expect to die before then) about the value of a cookie, then that sets a constraint on how to calculate expected utility.
Fair enough. Of course, there’s no law of physics ruling out Future Tuesday Indifference, either. We go by plausibility and elegance. Admittedly, “average the branches” looks about equally plausible and elegant to “sum the branches”, but I think the former becomes implausible when we look at cases where some of the branches are very short-lived.
Requiring that the ancestor and descendants agree is contrary to the spirit of allowing selfish preferences, I think, in the sense of “selfish” that you’ve defined. If Methuselah is selfish, Methuselah(1000AD) values the experience of Methuselah(900AD), who values the experience of Methuselah(800AD), but M1000 doesn’t value the experience of M800.
As the caveat goes, “The copies have to be people who you would actually like to be.” Dying quickly seems like it would really put a damper on the expected utility of being a copy. (Mathematically, the relevant utility here is a time-integral)
I don’t see why your claims about Methuselah follow, but I do agree that under this model, agents don’t care about their past self—they just do what causes them to have high expected utility. Strictly, this is possible independent of whether descendants and ancestors agree or disagree. But if self-modification is possible, such conflicting selfish preferences would get modified away into nonconflicting selfless preferences.
Not if the copy doesn’t anticipate dying. Perhaps all the copies go thru a brief dim-witted phase of warm happiness (and the original expects this), in which all they can think is “yup warm and happy, just like I expected”, followed by some copies dying and others recovering full intellect and living. Any of those copies is someone I’d “like to be” in the better-than-nothing sense. Is the caveat “like to be” a stronger sense?
I’m confused—if agents don’t value their past self, in what sense do they agree or disagree with what the past-self was valuing? In any case, please reverse the order of the Methuselah valuing of time-slices.
Edit: Let me elaborate a story to motivate my some-copies-dying posit. I want to show that I’m not just “gaming the system,” as you were concerned to avoid using your caveat.
I’m in one spaceship of a fleet of fast unarmed robotic spaceships. As I feared but planned for, an enemy fleet shows up. This spaceship will be destroyed, but I can make copies of myself in one to all of the many other ships. Each copy will spend 10 warm-and-fuzzy dim-witted minutes reviving from their construction. The space battle will last 5 minutes. The spaceship at the farthest remove from the enemy has about a 10% chance of survival. The next-farthest has a 9 point something percent chance—and so on. The enemy uses an indeterministic algorithm to chase/target ships, so these probabilities are almost independent. If I copy to all the ships in the fleet, I have a very high probability of survival. But the maximum average expected utility is gotten by copying to just one ship.
I’m tapping out, sorry.