Averaging makes a lot of sense if the number of agents is going to be increased and decreased in non-relevant ways.
Eg: you are an upload. Soon, you are going to experience eating a chocolate bar, then stubbing your toe, then playing a tough but intriguing game. During this time, you will be simulated on n computers, all running exactly the same program of you experiencing this, without any deviations. But n may vary from moment to moment. Should you be willing to pay to make n higher during pleasant experience or lower during unpleasant ones, given that you will never detect this change?
I think there are some rather significant assumptions underlying the idea that they are “non-relevant”. At the very least, if the agents were distinguishable, I think you should indeed be willing to pay to make n higher. On the other hand, if they’re indistinguishable then it’s a more difficult question, but the anthropic averaging I suggested in my previous comments leads to absurd results.
the anthropic averaging I suggested in my previous comments leads to absurd results.
The anthropic averaging leads to absurd results only because it wasn’t a utility function over states of the world. Under heads, it ranked 50%Roger+50%Jack differently from the average utility of those two worlds.
Averaging makes a lot of sense if the number of agents is going to be increased and decreased in non-relevant ways.
Eg: you are an upload. Soon, you are going to experience eating a chocolate bar, then stubbing your toe, then playing a tough but intriguing game. During this time, you will be simulated on n computers, all running exactly the same program of you experiencing this, without any deviations. But n may vary from moment to moment. Should you be willing to pay to make n higher during pleasant experience or lower during unpleasant ones, given that you will never detect this change?
I think there are some rather significant assumptions underlying the idea that they are “non-relevant”. At the very least, if the agents were distinguishable, I think you should indeed be willing to pay to make n higher. On the other hand, if they’re indistinguishable then it’s a more difficult question, but the anthropic averaging I suggested in my previous comments leads to absurd results.
What’s your proposal here?
The anthropic averaging leads to absurd results only because it wasn’t a utility function over states of the world. Under heads, it ranked 50%Roger+50%Jack differently from the average utility of those two worlds.