This is definitely an interesting topic (I recently listened to an interview with Joseph Heinrich, author of The WEIRDest People In The World). It’s a serious problem if you’re trying to find a unique True Morality and ground it in human intuitions, but if we’ve managed to move past that then there’s still an interesting point here, the philosophical problem just gets turned inwards.
Of course different humans are allowed to prefer different things and even have different preferences about preference aggregation—so if you’re trying to build some decisionmaking procedure that aggregates human preferences, rather than being able to delegate the choice of how to do that to some unique True Meta-Morality, you have to do the philosophical work of figuring out what you want out of this aggregation process.
This is definitely an interesting topic (I recently listened to an interview with Joseph Heinrich, author of The WEIRDest People In The World). It’s a serious problem if you’re trying to find a unique True Morality and ground it in human intuitions, but if we’ve managed to move past that then there’s still an interesting point here, the philosophical problem just gets turned inwards.
Of course different humans are allowed to prefer different things and even have different preferences about preference aggregation—so if you’re trying to build some decisionmaking procedure that aggregates human preferences, rather than being able to delegate the choice of how to do that to some unique True Meta-Morality, you have to do the philosophical work of figuring out what you want out of this aggregation process.