This needs a distinction between a prior that is “prior to all your knowledge” and prior that already takes into account your current knowledge and would be updated by future observations. I guess prior in the first sense could be seen as a fixed aspect of preference, while prior in the second sense reflects a situation where an action might be performed, so that it can be different in different situations with the same preference.
Thus, perfectly altruistic Alice should have a different prior in the second sense, taking into account Alice’s knowledge rather than Bob’s, but the same prior in the first sense, reflecting the same distribution of caring over possible worlds as Bob.
Bob’s prior in the first sense is not factual knowledge, it’s a description of which worlds Bob considers how important, so Alice can’t improve on it by knowing something that Bob doesn’t. A difference in priors in the first sense reflects different distributions of moral relevance associated with possibilities. When Alice knows something that Bob doesn’t, it is a statement about her priors in the second sense, not the first sense.
Thus, to the extent that Alice doesn’t assume Bob’s priors in the first sense, Alice doesn’t follow Bob’s preference, which would be a failure of perfect altruism. Alice’s prior doesn’t reflect different (or additional) knowledge, so its use would not be an improvement in the sense of Bob’s preference.
This needs a distinction between a prior that is “prior to all your knowledge” and prior that already takes into account your current knowledge and would be updated by future observations. I guess prior in the first sense could be seen as a fixed aspect of preference, while prior in the second sense reflects a situation where an action might be performed, so that it can be different in different situations with the same preference.
Thus, perfectly altruistic Alice should have a different prior in the second sense, taking into account Alice’s knowledge rather than Bob’s, but the same prior in the first sense, reflecting the same distribution of caring over possible worlds as Bob.
Why should Alice have the same distribution of caring as Bob?
My definition of prior was in the first sense.
Bob’s prior in the first sense is not factual knowledge, it’s a description of which worlds Bob considers how important, so Alice can’t improve on it by knowing something that Bob doesn’t. A difference in priors in the first sense reflects different distributions of moral relevance associated with possibilities. When Alice knows something that Bob doesn’t, it is a statement about her priors in the second sense, not the first sense.
Thus, to the extent that Alice doesn’t assume Bob’s priors in the first sense, Alice doesn’t follow Bob’s preference, which would be a failure of perfect altruism. Alice’s prior doesn’t reflect different (or additional) knowledge, so its use would not be an improvement in the sense of Bob’s preference.