Finally, we have identified causal mechanisms underpinning many old values, and found them repugnant.
This is exactly the kind of reasoning I mocked in the post.
No, you mocked finding the values themselves repugnant, not their underlying mechanisms. If we find out that a value only exists because of a historical accident plus status quo bias, and that any society where it wasn’t the status quo would reject it when it was explained to them, then we should reject that value.
All such desiderata get satisfied automatically if your comment was generated by your sincere volition and not something else :-)
The fact that my volition might just consist of a pointer to CEV does not seem like much of an argument for choosing it over CEV, given that my volition also includes lots of poorly-understood other stuff, which I won’t get a chance to inspect if there’s no extrapolation, and which is more likely to make things worse than to make them better. Also, consider the worst case scenario: I have a stroke shortly before the AI reads out my volition.
I think your arguments, if they worked, would prove way too much.
If we find out that a value only exists because of a historical accident plus status quo bias, and that any society where it wasn’t the status quo would reject it when it was explained to them, then we should reject that value.
This standard allows us to throw away all values not directly linked to inclusive genetic fitness, and maybe even those that are. There’s no objective morality.
The fact that my volition might just consist of a pointer to CEV does not seem like much of an argument for choosing it over CEV, given that my volition also includes lots of poorly-understood other stuff, which I won’t get a chance to inspect if there’s no extrapolation, and which is more likely to make things worse than to make them better.
This argument works just as well for defending concrete wishes (“volcano lair with catgirls”) over CEV.
If we find out that a value only exists because of a historical accident plus status quo bias, and that any society where it wasn’t the status quo would reject it when it was explained to them, then we should reject that value.
This standard allows us to throw away all values not directly linked to inclusive genetic fitness, and maybe even those that are. There’s no objective morality.
Huh? We must have a difference of definitions somewhere, because that’s not what I think my argument says at all.
The fact that my volition might just consist of a pointer to CEV does not seem like much of an argument for choosing it over CEV, given that my volition also includes lots of poorly-understood other stuff, which I won’t get a chance to inspect if there’s no extrapolation, and which is more likely to make things worse than to make them better.
This argument works just as well for defending concrete wishes (“volcano lair with catgirls”) over CEV.
No, it doesn’t. This was a counterargument to the could-be-a-pointer argument, not a root-level argument; and if you expand it out, it actually favors CEV over concrete wishes, not the reverse.
The could-be-a-pointer argument is that since one person’s volition might just be the desire to have CEV implemented, so that one person’s volition is at least as good as CEV. But this is wrong, because that person’s volition will also include lots of other stuff, which is substantially random and so at least some of it will be bad. So you need to filter (extrapolate) those desires to get only the good ones. One way we could filter them is by throwing out everything except for a few concrete wishes, but that is not the best possible filter because it will throw out many aspects of volition that are good (and probably also necessary for preventing disastrous misinterpretations of the concrete wishes).
If we find out that a value only exists because of a historical accident plus status quo bias, and that any society where it wasn’t the status quo would reject it when it was explained to them, then we should reject that value.
How confident are you that what’s left of our values, under that rule, would be enough to be called a volition at all?
No, you mocked finding the values themselves repugnant, not their underlying mechanisms. If we find out that a value only exists because of a historical accident plus status quo bias, and that any society where it wasn’t the status quo would reject it when it was explained to them, then we should reject that value.
The fact that my volition might just consist of a pointer to CEV does not seem like much of an argument for choosing it over CEV, given that my volition also includes lots of poorly-understood other stuff, which I won’t get a chance to inspect if there’s no extrapolation, and which is more likely to make things worse than to make them better. Also, consider the worst case scenario: I have a stroke shortly before the AI reads out my volition.
I think your arguments, if they worked, would prove way too much.
This standard allows us to throw away all values not directly linked to inclusive genetic fitness, and maybe even those that are. There’s no objective morality.
This argument works just as well for defending concrete wishes (“volcano lair with catgirls”) over CEV.
Huh? We must have a difference of definitions somewhere, because that’s not what I think my argument says at all.
No, it doesn’t. This was a counterargument to the could-be-a-pointer argument, not a root-level argument; and if you expand it out, it actually favors CEV over concrete wishes, not the reverse.
The could-be-a-pointer argument is that since one person’s volition might just be the desire to have CEV implemented, so that one person’s volition is at least as good as CEV. But this is wrong, because that person’s volition will also include lots of other stuff, which is substantially random and so at least some of it will be bad. So you need to filter (extrapolate) those desires to get only the good ones. One way we could filter them is by throwing out everything except for a few concrete wishes, but that is not the best possible filter because it will throw out many aspects of volition that are good (and probably also necessary for preventing disastrous misinterpretations of the concrete wishes).
How confident are you that what’s left of our values, under that rule, would be enough to be called a volition at all?