These assumptions … are enough to specify uniquely how utility should work across copying, deleting, and merging.
I’m not sure they are. Let us peek into the mind of our hero as he tries to relax before the copying operation by thinking about his plans for the next morning.
He plans to make his morning run a long one—a full 6K. The endorphin rush will feel good.
Both the run and the rush are indexical, in that all copies participate on their own. So far, so good for your axioms.
Then he will shower and dress. It will be Tuesday, so he will wear his favorite blue and gold tie—the one Maria gave him in San Pedro three years ago.
Some kinds of property cannot be copied. I suppose we can just ignore this and stipulate that neither original nor copies have any property, but …
When he crosses 52nd Street, he will help blind Mrs. Atkins across the street.
Only one version of him will be able to do this. I suppose all copies will be happy that Mrs. Atkins got across the street safely, but only one of them gets the fuzzies for having helped.
Then, in the coffee shop, he will joke with that cute girl who works at the law school. It is her turn to buy him coffee.
Help! People are embedded in a web of obligations and expectations. They have made and received commitments. The axioms need to either account for this kind of thing, or else stipulate that copied persons must leave all this baggage behind.
End of fable. I suppose that if I were to try to find the one essential point in this fable—the one thing that we can’t stipulate away, it would probably be found in the blind woman portion of the story. Even though all copies gain utility from Mrs. Atkins crossing the street, only one of them actually makes use of this utility in his decision making. And only one of them gets fuzzies. It seems to me that the axioms need to deal not just with how future utility is distributed, but also with how anticipated future decision points are distributed in the copying operation.
These assumptions … are enough to specify uniquely how utility should work across copying, deleting, and merging.
I’m not sure they are.
They are—in that given the assumptions, you have that result. Now, the result might not be nice or ideal, but that means that the assumptions are wrong.
Now, you are pointing out that my solution is counter-intuitive. I agree completely; my intuition was in the previous post, and it all went wrong. I feel these axioms are like those of expected utility—idealisations that you would want an AI to follow, that you might want to approximate, but that humans can’t follow.
But there is a mistake that a lot of utilitarians make, and that is to insist that their utility function must be simple. Not so; there is no reason to require that. Here, I’m sure we could deal with these problems in many different ways.
The blind woman example is the hardest. By assumption, the others will have to feel the fuzzies for a copy of them helping her cross the street, or a non-indexical happiness from Mrs. Atkins being helped across the street, or similar. But the others can easily be dealt with… The simplest way for obligations and expectations is to say that all copies have all the obligations and expectations incurred by each of their members. Legally at least, this seems perfectly fine.
As for the property, there are many solutions, and one extreme one is this: only the utility of the copy that has the tie/house/relationship actually matters. As I said, I am only forbidding intrinsic differences between copies; a setup that says “you must serve the copy of you that has the blue and gold tie” is perfectly possible. Though stupid. But most intuitive ways of doing things that you can come up with can be captured by the utility function, especially if you can make it non-indexical.
I like your axiomatic approach, but …
I’m not sure they are. Let us peek into the mind of our hero as he tries to relax before the copying operation by thinking about his plans for the next morning.
He plans to make his morning run a long one—a full 6K. The endorphin rush will feel good.
Both the run and the rush are indexical, in that all copies participate on their own. So far, so good for your axioms.
Then he will shower and dress. It will be Tuesday, so he will wear his favorite blue and gold tie—the one Maria gave him in San Pedro three years ago.
Some kinds of property cannot be copied. I suppose we can just ignore this and stipulate that neither original nor copies have any property, but …
When he crosses 52nd Street, he will help blind Mrs. Atkins across the street.
Only one version of him will be able to do this. I suppose all copies will be happy that Mrs. Atkins got across the street safely, but only one of them gets the fuzzies for having helped.
Then, in the coffee shop, he will joke with that cute girl who works at the law school. It is her turn to buy him coffee.
Help! People are embedded in a web of obligations and expectations. They have made and received commitments. The axioms need to either account for this kind of thing, or else stipulate that copied persons must leave all this baggage behind.
End of fable. I suppose that if I were to try to find the one essential point in this fable—the one thing that we can’t stipulate away, it would probably be found in the blind woman portion of the story. Even though all copies gain utility from Mrs. Atkins crossing the street, only one of them actually makes use of this utility in his decision making. And only one of them gets fuzzies. It seems to me that the axioms need to deal not just with how future utility is distributed, but also with how anticipated future decision points are distributed in the copying operation.
They are—in that given the assumptions, you have that result. Now, the result might not be nice or ideal, but that means that the assumptions are wrong.
Now, you are pointing out that my solution is counter-intuitive. I agree completely; my intuition was in the previous post, and it all went wrong. I feel these axioms are like those of expected utility—idealisations that you would want an AI to follow, that you might want to approximate, but that humans can’t follow.
But there is a mistake that a lot of utilitarians make, and that is to insist that their utility function must be simple. Not so; there is no reason to require that. Here, I’m sure we could deal with these problems in many different ways.
The blind woman example is the hardest. By assumption, the others will have to feel the fuzzies for a copy of them helping her cross the street, or a non-indexical happiness from Mrs. Atkins being helped across the street, or similar. But the others can easily be dealt with… The simplest way for obligations and expectations is to say that all copies have all the obligations and expectations incurred by each of their members. Legally at least, this seems perfectly fine.
As for the property, there are many solutions, and one extreme one is this: only the utility of the copy that has the tie/house/relationship actually matters. As I said, I am only forbidding intrinsic differences between copies; a setup that says “you must serve the copy of you that has the blue and gold tie” is perfectly possible. Though stupid. But most intuitive ways of doing things that you can come up with can be captured by the utility function, especially if you can make it non-indexical.