It would be completely justified in killing other people and taking their things for itself, or enslaving them for its own benefit.
I thought the standard solution is to disregard the components of the Utility Monster’s utility that are harmful to others, directly or indirectly. Even unbounded joy you may gain from torturing me does not require me to submit and suffer. See also the comments to my old post on Jews and Nazis.
That solution doesn’t work. If consuming resources is counted as “harmful to others”, then you end up saying that it should starve (with no good distinction with which to say that others shouldn’t). If consuming resources doesn’t count as harmful to others, then you end up giving it the whole universe. You want to give it some, but not all, of the resources. If you try to use references to property distinctions inside the utility function to do that, you’ve disqualified your utility function from the role of distinguishing good and bad legal and economic systems, and epistemology explodes.
You want to give it some, but not all, of the resources.
Right, one has to arbitrate between harmful components of various utility monsters (UM) (which most people are in the approximation of limited resources) somehow. But you should not need to kill or torture people just because the UM enjoys it a lot.
Now, how to optimize harmful preferences? If there are enough resources to saturate every non-UM utility, then there is no problem. If there isn’t enough, the linear programming approach would reduce every non-UM to “life barely worth celebrating” and give the rest to the hungriest UM. Whether this is a good solution, I do not know.
If you try to use references to property distinctions inside the utility function to do that, you’ve disqualified your utility function from the role of distinguishing good and bad legal and economic systems, and epistemology explodes.
I did not follow that, feel free to give an example.
But you should not need to kill or torture people just because the UM enjoys it a lot.
Again, I agree, but I did not mention that in the OP because many people would not have read our previous discussion, and might have been confused when I suddenly went off on a tangent about how “malicious preferences shouldn’t count” in an essay on a totally different subject.
The relevant question then, is how should we split resources between the monster and between other people when attempting to satisfy preferences that do not involve harming other as an end in itself.
If there are enough resources to saturate every non-UM utility, then there is no problem.
I know I was the one who started using the word “saturate” in the first place, but after some thought “satisfice” is a much better approximation of what I meant.
I did not follow that, feel free to give an example.
I think he is arguing that someone might try to get out of giving the monster resources by claiming that the other people in the world own their share of resources, and that it is bad to take private property. The problem with this is that since property is a legal construct, one can simply argue that property rights should be abolished for the Monster’s sake. If one tries to claim that property rights somehow transcend other utility concerns that means your utility function does not make any distinction between what kinds of property rights are good and which are bad,
I don’t know why this makes epistemology explode either.
Also, I don’t think you ever made such an argument in the first place, he was probably just mentioning it for completeness’ sake.
That would definitely work if the utility monster wanted to kill or enslave people just for kicks. I still stand by the idea we hammered out in the discussion there. Malicious preference shouldn’t count. The real problem arises when considering on how to split the various resources in the society between the monster and its inhabitants so they can fulfill their “neutral” preferences (I am using the terminology from that discussion).
The Utility Monster problem suggests the idea that one individual might be so good at using resources to satisfy its “neutral” preferences that it would be better to give all the resources to it instead of sharing them among everyone.
Let’s suppose the monster enjoys various activities that are “neutral,” they are not directly harmful to other people. But they do use up resources. How many resources should be given to the monster and how many should be given to other people? Should we give the monster everything and let everyone else starve to death, because it will get so much more enjoyment out of them? Should everyone be given else just enough resources to live a life barely worth living, and then give everything else to the monster? Should everyone else be given resources until they reach their satiation point and then the monster gets the rest?
It seems wrong to give everything to the monster, even though that would result in the most satisfied neutral preferences. It also seems wrong to give people just enough for lives barely worth living. It seems best to me to share so that everyone can live great lives, even if giving stuff to the monster would be best.
Limited resource use should be counted as indirect harm, surely.
If there are a finite amount of resources then you harm other people just by existing, because by using resources to live you are reducing the amount available for other people to use. By “limited” do you mean “resource use above a certain threshold?” What would that threshold be? Would it change depending on how many resources a given society has?
Are you suggesting that everyone is entitled to a certain level of life quality, and that any desires that would reduce that level of life quality if fulfilled should count as “malicious?” That is a line of thought that hadn’t fully occurred to me. It seems to have some similarities with prioritarianism.
You mean “barely worth celebrating”, surely?
Yes. I used the other term in the OP because I thought not everyone who read it would have read Eliezer’s essay and got stuck in the habit.
EDIT: When I said “you harm other people just by existing” that technically isn’t true in the present because we live in a non-malthusian world with a growing economy. Adding more people actually increases the amount of resources available to everyone because there are more people to do work. Assume, for the sake of the argument, that in this thought experiment the amount of resources available to a society is fixed.
I don’t think so, for any strong version of ‘standard.’ There’s a simple modification where instead of the monster eating everyone else, the monster eats all the food that would have sustained everyone else, on triage grounds.
I thought the standard solution is to disregard the components of the Utility Monster’s utility that are harmful to others, directly or indirectly. Even unbounded joy you may gain from torturing me does not require me to submit and suffer. See also the comments to my old post on Jews and Nazis.
That solution doesn’t work. If consuming resources is counted as “harmful to others”, then you end up saying that it should starve (with no good distinction with which to say that others shouldn’t). If consuming resources doesn’t count as harmful to others, then you end up giving it the whole universe. You want to give it some, but not all, of the resources. If you try to use references to property distinctions inside the utility function to do that, you’ve disqualified your utility function from the role of distinguishing good and bad legal and economic systems, and epistemology explodes.
Right, one has to arbitrate between harmful components of various utility monsters (UM) (which most people are in the approximation of limited resources) somehow. But you should not need to kill or torture people just because the UM enjoys it a lot.
Now, how to optimize harmful preferences? If there are enough resources to saturate every non-UM utility, then there is no problem. If there isn’t enough, the linear programming approach would reduce every non-UM to “life barely worth celebrating” and give the rest to the hungriest UM. Whether this is a good solution, I do not know.
I did not follow that, feel free to give an example.
Again, I agree, but I did not mention that in the OP because many people would not have read our previous discussion, and might have been confused when I suddenly went off on a tangent about how “malicious preferences shouldn’t count” in an essay on a totally different subject.
The relevant question then, is how should we split resources between the monster and between other people when attempting to satisfy preferences that do not involve harming other as an end in itself.
I know I was the one who started using the word “saturate” in the first place, but after some thought “satisfice” is a much better approximation of what I meant.
I think he is arguing that someone might try to get out of giving the monster resources by claiming that the other people in the world own their share of resources, and that it is bad to take private property. The problem with this is that since property is a legal construct, one can simply argue that property rights should be abolished for the Monster’s sake. If one tries to claim that property rights somehow transcend other utility concerns that means your utility function does not make any distinction between what kinds of property rights are good and which are bad,
I don’t know why this makes epistemology explode either.
Also, I don’t think you ever made such an argument in the first place, he was probably just mentioning it for completeness’ sake.
How does this have anything to do with epistemology?
That would definitely work if the utility monster wanted to kill or enslave people just for kicks. I still stand by the idea we hammered out in the discussion there. Malicious preference shouldn’t count. The real problem arises when considering on how to split the various resources in the society between the monster and its inhabitants so they can fulfill their “neutral” preferences (I am using the terminology from that discussion).
The Utility Monster problem suggests the idea that one individual might be so good at using resources to satisfy its “neutral” preferences that it would be better to give all the resources to it instead of sharing them among everyone.
Let’s suppose the monster enjoys various activities that are “neutral,” they are not directly harmful to other people. But they do use up resources. How many resources should be given to the monster and how many should be given to other people? Should we give the monster everything and let everyone else starve to death, because it will get so much more enjoyment out of them? Should everyone be given else just enough resources to live a life barely worth living, and then give everything else to the monster? Should everyone else be given resources until they reach their satiation point and then the monster gets the rest?
It seems wrong to give everything to the monster, even though that would result in the most satisfied neutral preferences. It also seems wrong to give people just enough for lives barely worth living. It seems best to me to share so that everyone can live great lives, even if giving stuff to the monster would be best.
What I said is
Limited resource use should be counted as indirect harm, surely. Now, the problem is how to arbitrate between multiple Resource Monsters.
I do not see any immediate problem with this approach.
You mean “barely worth celebrating”, surely?
If there are a finite amount of resources then you harm other people just by existing, because by using resources to live you are reducing the amount available for other people to use. By “limited” do you mean “resource use above a certain threshold?” What would that threshold be? Would it change depending on how many resources a given society has?
Are you suggesting that everyone is entitled to a certain level of life quality, and that any desires that would reduce that level of life quality if fulfilled should count as “malicious?” That is a line of thought that hadn’t fully occurred to me. It seems to have some similarities with prioritarianism.
Yes. I used the other term in the OP because I thought not everyone who read it would have read Eliezer’s essay and got stuck in the habit.
EDIT: When I said “you harm other people just by existing” that technically isn’t true in the present because we live in a non-malthusian world with a growing economy. Adding more people actually increases the amount of resources available to everyone because there are more people to do work. Assume, for the sake of the argument, that in this thought experiment the amount of resources available to a society is fixed.
I don’t think so, for any strong version of ‘standard.’ There’s a simple modification where instead of the monster eating everyone else, the monster eats all the food that would have sustained everyone else, on triage grounds.