I don’t think that people valuing eternal torture of other humans is much of a concern, because they don’t value it nearly as much as the people in question disvalue being tortured. I bet there are a lot more people who care about animals’ feelings and who care a lot more, than those who care about the aesthetics of brutality in nature. I think the majority of people have more instincts of concern for animals than their actions suggest, because now it is convenient to screw over animals as an externality of eating tasty food, and the animals suffering are out of sight, and the connection between buying meat and animals living terrible lives elsewhere is hard to visualize. The same population that buys meat from farms that treat animals to awful lives also enacts animal cruelty laws. As evidence that caring more about animals is something that would be strengthened by thinking more about, consider the results of the 2012 LessWrong Survey:
(Though some of this is probably due to lesswrongers being richer than the average american (something that’s probably true because wealthy people more time to read about abstruse topics on the internet))
The biggest peculiarity of Brian Tomasik’s utility function, that is least likely to ever be shared by the majority of humanity, is probably not that he cares about animals (even that he cares about insects) but that he cares so much more about suffering than happiness and other good things. (I am basing this assessment of his utility function on a post of his I read on http://www.felicifia.org a while ago, which I can’t find now).
The exchange rate in your utility function between good things and bad things is pretty relevent to whether you should prefer CEV or paperclipping (And what the changes in the probabilities of each even based on actions you might take would have to be in order justify them) and whether you think lab universes would be a good thing.
So if you are not a negative utilitarian, or nearly one, even if Brian Tomasik’s beliefs about matters of fact are very correlated with reality, be careful of his policy recommendations.
I bet there are a lot more people who care about animals’ feelings and who care a lot more, than those who care about the aesthetics of brutality in nature.
Well, at the moment, there are hundreds of environmental-preservation organizations and basically no organizations dedicated to reducing wild-animal suffering. Environmentalism as a cause is much more mainstream than animal welfare. Just like the chickens that go into people’s nuggets, animals suffering in nature “are out of sight, and the connection between [preserving pristine habitats] and animals living terrible lives elsewhere is hard to visualize.”
It’s encouraging that more LessWrongers are veg than average, although I think 12.4% is pretty typical for elite universities and the like as well. (But maybe that underscores your point.)
The biggest peculiarity of Brian Tomasik’s utility function, that is least likely to ever be shared by the majority of humanity, is probably not that he cares about animals (even that he cares about insects) but that he cares so much more about suffering than happiness and other good things.
An example post. I care a lot about suffering, a little about happiness, and none about other things.
The exchange rate in your utility function between good things and bad things is pretty relevent to whether you should prefer CEV or paperclipping (And what the changes in the probabilities of each even based on actions you might take would have to be in order justify them) and whether you think lab universes would be a good thing.
I don’t think that people valuing eternal torture of other humans is much of a concern, because they don’t value it nearly as much as the people in question disvalue being tortured.
Suppose most people agree on valuing the torture of a few people, and only a few people disagree. Would you be OK with the majority’s values outweighing the minority’s, if it’s a large enough majority?
If you’re OK with that, and if this is not specific to the example of torture, then you are effectively saying that you value the extrapolated consensus values of humanity more than your own, even though you don’t know what those values may be. That you value the (unspecified) CEV process, and whatever values it ends up generating, more than any other values you currently hold. Is that so?
Even if you’re OK with that, you’d be vulnerable to a “clone utility monster”: if I can clone myself faster than average, then the values of me and my clones will come to dominate the global population. This seems true for almost any value aggregation process given a large enough majority (fast enough cloning).
No, I would not be okay with it.
I don’t terminally value CEV. I think it would be instrumentally valuable, because scenarios where everyone wants to torture a few people are not that likely. I would prefer that only my own extrapolated utility function controlled the universe. Unlike Eliezer Yudkowsky, I don’t care that much about not being a jerk. But that is not going to happen.
If this detail from the original paper still stands, the CEV is allowed to modify the extrapolating process. So if there was the threat of everyone having to race to clone themselves as much as possible for more influence, it might modify itself to give clones less weight, or prohibit cloning.
So if there was the threat of everyone having to race to clone themselves as much as possible for more influence, it might modify itself to give clones less weight, or prohibit cloning
Prohibiting these things, and CEV self-modifying in general, means optimizing for certain values or a certain outcome. Where do these values come from? From the CEV’s programmers. But if you let certain predetermined values override the (unknown) CEV-extrapolated values, how do you make these choices, and where do you draw the line?
I mean that the CEV extrapolated from the entire population before they start a clone race could cause that self-modification or prohibition, not something explicitly put in by the programmers.
I don’t think that people valuing eternal torture of other humans is much of a concern, because they don’t value it nearly as much as the people in question disvalue being tortured. I bet there are a lot more people who care about animals’ feelings and who care a lot more, than those who care about the aesthetics of brutality in nature. I think the majority of people have more instincts of concern for animals than their actions suggest, because now it is convenient to screw over animals as an externality of eating tasty food, and the animals suffering are out of sight, and the connection between buying meat and animals living terrible lives elsewhere is hard to visualize. The same population that buys meat from farms that treat animals to awful lives also enacts animal cruelty laws. As evidence that caring more about animals is something that would be strengthened by thinking more about, consider the results of the 2012 LessWrong Survey:
(Though some of this is probably due to lesswrongers being richer than the average american (something that’s probably true because wealthy people more time to read about abstruse topics on the internet))
The biggest peculiarity of Brian Tomasik’s utility function, that is least likely to ever be shared by the majority of humanity, is probably not that he cares about animals (even that he cares about insects) but that he cares so much more about suffering than happiness and other good things. (I am basing this assessment of his utility function on a post of his I read on http://www.felicifia.org a while ago, which I can’t find now).
The exchange rate in your utility function between good things and bad things is pretty relevent to whether you should prefer CEV or paperclipping (And what the changes in the probabilities of each even based on actions you might take would have to be in order justify them) and whether you think lab universes would be a good thing.
So if you are not a negative utilitarian, or nearly one, even if Brian Tomasik’s beliefs about matters of fact are very correlated with reality, be careful of his policy recommendations.
Well, at the moment, there are hundreds of environmental-preservation organizations and basically no organizations dedicated to reducing wild-animal suffering. Environmentalism as a cause is much more mainstream than animal welfare. Just like the chickens that go into people’s nuggets, animals suffering in nature “are out of sight, and the connection between [preserving pristine habitats] and animals living terrible lives elsewhere is hard to visualize.”
It’s encouraging that more LessWrongers are veg than average, although I think 12.4% is pretty typical for elite universities and the like as well. (But maybe that underscores your point.)
An example post. I care a lot about suffering, a little about happiness, and none about other things.
Yep!
Suppose most people agree on valuing the torture of a few people, and only a few people disagree. Would you be OK with the majority’s values outweighing the minority’s, if it’s a large enough majority?
If you’re OK with that, and if this is not specific to the example of torture, then you are effectively saying that you value the extrapolated consensus values of humanity more than your own, even though you don’t know what those values may be. That you value the (unspecified) CEV process, and whatever values it ends up generating, more than any other values you currently hold. Is that so?
Even if you’re OK with that, you’d be vulnerable to a “clone utility monster”: if I can clone myself faster than average, then the values of me and my clones will come to dominate the global population. This seems true for almost any value aggregation process given a large enough majority (fast enough cloning).
No, I would not be okay with it. I don’t terminally value CEV. I think it would be instrumentally valuable, because scenarios where everyone wants to torture a few people are not that likely. I would prefer that only my own extrapolated utility function controlled the universe. Unlike Eliezer Yudkowsky, I don’t care that much about not being a jerk. But that is not going to happen. If this detail from the original paper still stands, the CEV is allowed to modify the extrapolating process. So if there was the threat of everyone having to race to clone themselves as much as possible for more influence, it might modify itself to give clones less weight, or prohibit cloning.
Prohibiting these things, and CEV self-modifying in general, means optimizing for certain values or a certain outcome. Where do these values come from? From the CEV’s programmers. But if you let certain predetermined values override the (unknown) CEV-extrapolated values, how do you make these choices, and where do you draw the line?
I mean that the CEV extrapolated from the entire population before they start a clone race could cause that self-modification or prohibition, not something explicitly put in by the programmers.