had magicked house elves into existence [...] should we change them?
I’m explicitly arguing that even though we might not want to change them, we could still prefer they not exist in the first place.
should we change from our world to one where people are not culturally molded to appreciate any of our current values?
I’m trying to synthesise actual human values, not hypothetical other values that other beings might have. So in this process, our current values (or our current meta-preferences for our future values) get special place. If we had different values currently, the synthesis would be different. So that would-change is, from our perspective, a loss.
Agreed. I’m just trying to think through why we should / should not privilege the status quo. I notice I’m confused about this, since the reversal heuristic implies we shouldn’t. If we take this approach to an extreme, aren’t we locking in the status-quo as a base for allowing only pareto improvements, rather than overall utilitarian gains?
(I’ll note that Eric Drexler’s Pareto-topia argument explicitly allows for this condition—I’m just wondering whether it is ideal, or a necessary compromise.)
I’m trying to synthesise actual human values, not hypothetical other values that other beings might have.
To be clear, when you say “actual human values”, do you mean anything different than just “the values of the humans alive today, in the year 2019″? You mention “other beings”—is this meant to include other humans in the past who might have held different values?
The aim is to be even more specific—the values of a specific human at a specific time. Then what we do with these syntheses is another point, how much change to allow, etc… Including other humans in the past is a choice that we then need to make, or not.
I see, thank you. So then, would you say this doesn’t & isn’t intended to answer any question like “whose perspective should be taken into account?”, but that it instead assumes some answer to that question has already been specified, & is meant to address what to do given this chosen perspective?
I’m explicitly arguing that even though we might not want to change them, we could still prefer they not exist in the first place.
I’m trying to synthesise actual human values, not hypothetical other values that other beings might have. So in this process, our current values (or our current meta-preferences for our future values) get special place. If we had different values currently, the synthesis would be different. So that would-change is, from our perspective, a loss.
Agreed. I’m just trying to think through why we should / should not privilege the status quo. I notice I’m confused about this, since the reversal heuristic implies we shouldn’t. If we take this approach to an extreme, aren’t we locking in the status-quo as a base for allowing only pareto improvements, rather than overall utilitarian gains?
(I’ll note that Eric Drexler’s Pareto-topia argument explicitly allows for this condition—I’m just wondering whether it is ideal, or a necessary compromise.)
It’s locking in the moral/preference status quo; once that’s done, non-Pareto overall gains are fine.
Even when locking in that status quo, it explicitly trades off certain values against others, so there is no “only Pareto” restriction.
I have a research agenda to be published soon that will look into these issues in more detail.
To be clear, when you say “actual human values”, do you mean anything different than just “the values of the humans alive today, in the year 2019″? You mention “other beings”—is this meant to include other humans in the past who might have held different values?
The aim is to be even more specific—the values of a specific human at a specific time. Then what we do with these syntheses is another point, how much change to allow, etc… Including other humans in the past is a choice that we then need to make, or not.
I see, thank you. So then, would you say this doesn’t & isn’t intended to answer any question like “whose perspective should be taken into account?”, but that it instead assumes some answer to that question has already been specified, & is meant to address what to do given this chosen perspective?
It doesn’t intend to answer those questions; but those questions become a lot easier to answer once this issue is solved.