Agreed. I’m just trying to think through why we should / should not privilege the status quo. I notice I’m confused about this, since the reversal heuristic implies we shouldn’t. If we take this approach to an extreme, aren’t we locking in the status-quo as a base for allowing only pareto improvements, rather than overall utilitarian gains?
(I’ll note that Eric Drexler’s Pareto-topia argument explicitly allows for this condition—I’m just wondering whether it is ideal, or a necessary compromise.)
Agreed. I’m just trying to think through why we should / should not privilege the status quo. I notice I’m confused about this, since the reversal heuristic implies we shouldn’t. If we take this approach to an extreme, aren’t we locking in the status-quo as a base for allowing only pareto improvements, rather than overall utilitarian gains?
(I’ll note that Eric Drexler’s Pareto-topia argument explicitly allows for this condition—I’m just wondering whether it is ideal, or a necessary compromise.)
It’s locking in the moral/preference status quo; once that’s done, non-Pareto overall gains are fine.
Even when locking in that status quo, it explicitly trades off certain values against others, so there is no “only Pareto” restriction.
I have a research agenda to be published soon that will look into these issues in more detail.